Gravity-aligned Rotation Averaging with Circular Regression
Extracted source text · arXiv 2410.12763
% \documentclass[runningheads]{llncs}
% % \usepackage[review,year=2024,ID=5651]{eccv}
% % \usepackage[mobile,year=2024,ID=5651]{eccv}
\documentclass[runningheads]{llncs}
% ---------------------------------------------------------------
% Include basic ECCV package
% TODO REVIEW: Insert your submission number below by replacing '*****'
% TODO FINAL: Comment out the following line for the camera-ready version
% \usepackage[review,year=2024,ID=5651]{eccv}
% TODO FINAL: Un-comment the following line for the camera-ready version
% \usepackage{eccv}
% OPTIONAL: Un-comment the following line for a version which is easier to read
% on small portrait-orientation screens (e.g., mobile phones, or beside other windows)
\usepackage[mobile]{eccv}
% Other packages
% Commonly used abbreviations (\eg, \ie, \etc, \cf, \etal, etc.)
\usepackage{eccvabbrv}
\usepackage{graphicx}
\usepackage[accsupp]{axessibility}
% Hyperref package
\usepackage{bm}
\usepackage{amsfonts}
\usepackage{textcomp}
\usepackage{stmaryrd}
%\usepackage{algorithm}
%\usepackage{algorithmic}
\usepackage[ruled,vlined]{algorithm2e}
%\usepackage[ruled,vlined]{algorithm2e}
\input{macros}
\DeclareMathOperator*{\argmax}{argmax}
\DeclareMathOperator*{\argmin}{argmin}
% The "axessiblity" package can be found at: https://ctan.org/pkg/axessibility?lang=en
\usepackage[accsupp]{axessibility} % Improves PDF readability for those with disabilities.
% ---------------------------------------------------------------
% Hyperref package
% It is strongly recommended to use hyperref, especially for the review version.
% Please disable hyperref *only* if you encounter grave issues.
% hyperref with option pagebackref eases the reviewers' job, but should be disabled for the final version.
%
% If you comment hyperref and then uncomment it, you should delete
% main.aux before re-running LaTeX.
% (Or just hit 'q' on the first LaTeX run, let it finish, and you
% should be clear).
% TODO FINAL: Comment out the following line for the camera-ready version
% \usepackage[pagebackref,breaklinks,colorlinks,citecolor=eccvblue]{hyperref}
% TODO FINAL: Un-comment the following line for the camera-ready version
\usepackage{hyperref}
% Support for ORCID icon
\usepackage{orcidlink}
%% Latex cheat code for 8 pages
\setlength{\floatsep}{6pt plus2pt minus2pt}
\setlength{\textfloatsep}{6pt plus2pt minus2pt}
\setlength{\dblfloatsep}{6pt plus2pt minus2pt}
\setlength{\dbltextfloatsep}{6pt plus2pt minus2pt}
\begin{document}
% ---------------------------------------------------------------
% TODO REVIEW: Replace with your title
\title{Gravity-aligned Rotation Averaging with\\ Circular Regression}
% TODO REVIEW: If the paper title is too long for the running head, you can set
% an abbreviated paper title here. If not, comment out.
% \titlerunning{Abbreviated paper title}
% TODO FINAL: Replace with your author list.
% Include the authors' OCRID for the camera-ready version, if at all possible.
% \author{First Author\inst{1}\orcidlink{0000-1111-2222-3333} \and
% Second Author\inst{2,3}\orcidlink{1111-2222-3333-4444} \and
% Third Author\inst{3}\orcidlink{2222--3333-4444-5555}}
\author{Linfei Pan$^1$
\and Marc Pollefeys$^{1,2}$
\and Dániel Baráth$^1$\\
$^1$ETH Zurich
\quad\quad $^2$Microsoft\\
\texttt{\{linfei.pan, marc.pollefeys, danielbela.barath\}@inf.ethz.ch}
}
% \author{Linfei Pan\inst{1} \and
% Marc Pollefeys\inst{1,2} \and
% Daniel Barath\inst{1}}
% TODO FINAL: Replace with an abbreviated list of authors.
\authorrunning{L.~Pan et al.}
% First names are abbreviated in the running head.
% If there are more than two authors, 'et al.' is used.
% % % TODO FINAL: Replace with your institution list.
\institute{
% ETH Zurich
% \and
% Microsoft
}
% \institute{Princeton University, Princeton NJ 08544, USA \and
% Springer Heidelberg, Tiergartenstr.~17, 69121 Heidelberg, Germany
% \email{lncs@springer.com}\\
% \url{http://www.springer.com/gp/computer-science/lncs} \and
% ABC Institute, Rupert-Karls-University Heidelberg, Heidelberg, Germany\\
% \email{\{abc,lncs\}@uni-heidelberg.de}}
\maketitle
%%%%%%%%% ABSTRACT
\begin{abstract}
Reconstructing a 3D scene from unordered images is pivotal in computer vision and robotics, with applications spanning crowd-sourced mapping and beyond. While global Structure-from-Motion (SfM) techniques are scalable and fast, they often compromise on accuracy. To address this, we introduce a principled approach that integrates gravity direction into the rotation averaging phase of global pipelines, enhancing camera orientation accuracy and reducing the degrees of freedom. This additional information is commonly available in recent consumer devices, such as smartphones, mixed-reality devices and drones, making the proposed method readily accessible. Rooted in circular regression, our algorithm has similar convergence guarantees as linear regression. It also supports scenarios where only a subset of cameras have known gravity. Additionally, we propose a mechanism to refine error-prone gravity. We achieve state-of-the-art accuracy on four large-scale datasets. Particularly, the proposed method improves upon the SfM baseline by 13 AUC@$1^\circ$ points, on average, while running eight times faster. It also outperforms the standard planar pose graph optimization technique by 23 AUC@$1^\circ$ points. The code is at \url{https://github.com/colmap/glomap}.
% The code will be made publicly available.
\end{abstract}
%%%%%%%%% BODY TEXT
\input{tex/text/introduction}
\input{tex/text/related_work}
\input{tex/text/method}
\input{tex/text/experiments}
\input{tex/text/conclusion}
\section*{Acknowledgment}
The authors thank Mihai Dusmanu and Paul‑Edouard Sarlin for their help on the LaMAR dataset.
We also want to thank Yunke Ao and Shaohui Liu for helping to review the paper.
This work was partially funded by the Hasler Stiftung Research Grant via the ETH Zurich Foundation and the ETH Zurich Career Seed Award.
Linfei Pan was supported by gift funding from Microsoft.
% ---- Bibliography ----
%
% BibTeX users should specify bibliography style 'splncs04'.
% References will then be sorted and formatted in the correct style.
%
\bibliographystyle{splncs04}
\bibliography{main}
%%%%%%%%%% Merge with supplemental materials %%%%%%%%%%
\newpage
\begin{center}
\textbf{\large Supplemental Materials}
\end{center}
%%%%%%%%%% Merge with supplemental materials %%%%%%%%%%
%%%%%%%%%% Prefix a "S" to all equations, figures, tables and reset the counter %%%%%%%%%%
\setcounter{equation}{0}
\setcounter{figure}{0}
\setcounter{table}{0}
\setcounter{section}{0}
\makeatletter
\renewcommand{\theequation}{S\arabic{equation}}
\renewcommand{\thefigure}{S\arabic{figure}}
\renewcommand{\thetable}{S\arabic{table}}
\renewcommand{\thesection}{S\arabic{section}}
%%%%%%%%%% Prefix a "S" to all equations, figures, tables and reset the counter %%%%%%%%%%
\input{tex/text/appendix}
\end{document}
% \usepackage[nointegrals]{wasysym}
\usepackage{booktabs}
\usepackage{balance}
\usepackage{multirow}
\usepackage{overpic}
% \usepackage{bbm}
% \usepackage{dsfont}
\usepackage{cite}
\usepackage[font=small,labelfont=bf]{caption}
\usepackage{color}
% \usepackage[table]{xcolor}
\usepackage{pifont}
% \usepackage{etoolbox}
%\usepackage{float}
% \makeatletter
% \@namedef{ver@everyshi.sty}{}
% \makeatother
\usepackage{tikz}
\usetikzlibrary{patterns}
\usepackage{colortbl}
\usepackage{pgfplots}
\pgfplotsset{compat=1.17}
\usepackage{tabularx}
\usepackage{arydshln}
\usepackage{amssymb}% http://ctan.org/pkg/amssymb
\usepackage{pifont}% http://ctan.org/pkg/pifont
% \usepackage{times}
% \usepackage{epsfig}
% \usepackage{graphicx}
\usepackage{amsmath}
% \usepackage{amssymb}
% \usepackage{comment}
%\usepackage{MnSymbol}
\usepackage{wasysym}%
% \usepackage{soul}
% \usepackage{listings}
% \usepackage{lscape}
% \usepackage{rotating}
\usepackage{enumitem}
\usepackage{wrapfig}
\usepackage{xspace}
\usepackage{tabu}
\usepackage[super]{nth}
% \usepackage{subfigure}
% \usepackage{subcaption}
% \usepackage{wrapfig}
% \usepackage{floatrow}
% \newfloatcommand{capbtabbox}{table}[][\FBwidth]
\usepackage{subcaption}
\usepackage{scalefnt}
\usepackage{dsfont}
\usepackage{graphicx,calc}
% \usepackage{color}
% % for tikzplot
% \usepackage[utf8]{inputenc}
% \DeclareUnicodeCharacter{2212}{−}
% \usepgfplotslibrary{groupplots,dateplot}
% \usetikzlibrary{patterns,shapes.arrows}
% \pgfplotsset{compat=newest}
% % for tikzplot
% Colors ---------------------------------------
\definecolor{darkgreen}{RGB}{0,153,51}
\definecolor{linkgreen}{RGB}{52,130,48}
\definecolor{LightCyan}{rgb}{0.87,0.92,0.96}
% \definecolor{palette_sinopia}{RGB}{0,84,159}
% \definecolor{palette_green}{RGB}{132,177,88}
% \definecolor{my_orange}{HTML}{DEC402}
% \definecolor{my_blue}{HTML}{1F77B4}
% \definecolor{m_green}{RGB}{212, 251, 121}
% \definecolor{m_red}{RGB}{255, 126, 121}
% \definecolor{m_blue}{RGB}{118, 214, 255}
\definecolor{m_green}{RGB}{233, 254, 187}
\definecolor{m_orange}{RGB}{255, 212, 121}
\definecolor{m_red}{RGB}{255, 190, 188}
\definecolor{m_violet}{RGB}{215, 131, 255}
\definecolor{m_blue}{RGB}{186, 234, 255}
\definecolor{m_brown}{RGB}{255,212,120}
\definecolor{m_lightgreen}{RGB}{212,251,122}
\definecolor{notetext}{rgb}{0.7,0,0}
\definecolor{model_pink}{RGB}{235, 106, 164}
\definecolor{model_orange}{RGB}{250, 194, 122}
\definecolor{model_green}{RGB}{164, 210, 162}
\definecolor{model_gray}{RGB}{120, 120, 120}
\definecolor{model_yellow}{RGB}{251, 231, 171}
\definecolor{model_purple}{RGB}{190, 146, 211}
% Commands ---------------------------------------
\newcommand{\colorsquare}[1]{\hspace{-1pt}{\color{#1}$\blacksquare$}\hspace{-6.0pt}$\square$}
\newcommand{\colorcircle}[1]{{\color{#1}$\blackcirc$}\hspace{-6.4pt}$\circ$}
\newcommand{\name}{SceneGraphLoc}
\newcommand{\todo}[1]{{\color{red}#1}}
\newcommand{\refsec}[1]{Sec.\,\ref{sec:#1}}
\newcommand{\refchap}[1]{Chapter\,\ref{chp:#1}}
\newcommand{\refpart}[1]{Part\,\ref{part:#1}}
\newcommand{\refeq}[1]{Eq.\,\ref{eq:#1}}
\newcommand{\reffig}[1]{Fig.\,\ref{fig:#1}}
\newcommand{\reftab}[1]{Tab.\,\ref{tab:#1}}
\newcommand{\cmark}{\ding{51}}%
\newcommand{\xmark}{\ding{55}}%
\newcommand{\ColorMapCircle}[1]{\textcolor{#1}{\ding{108}}}
\newcommand\blfootnote[1]{%
\begingroup
\renewcommand\thefootnote{}\footnote{#1}%
\addtocounter{footnote}{-1}%
\endgroup
}
% Abbreviation ---------------------------------------
\def\eg{\emph{e.g.}\@\xspace} \def\Eg{\emph{E.g.}\@\xspace}
\def\ie{\emph{i.e.}\@\xspace} \def\Ie{\emph{I.e.}\@\xspace}
\def\cf{\emph{c.f.}\@\xspace} \def\Cf{\emph{C.f.}\@\xspace}
\def\etc{\emph{etc.}\@\xspace} \def\vs{\emph{vs.}\@\xspace}
\def\etal{\emph{et al.}\@\xspace}
%\newcommand{\colorbox}[1]{\fcolorbox{black}{#1}{\null}}
\newcommand{\circlenum}[1]{{\textcircled{\scriptsize{#1}}}}
\newcommand{\parag}[1]{\vspace{-6px} \vskip8pt \noindent \textbf{#1}}
\newcommand{\PAR}[1]{\vskip4pt \noindent {\bf #1~}}
\newcommand{\PARbegin}[1]{\noindent {\bf #1~}}
\newcommand{\paragt}[1]{\vspace{0px}\paragraph{#1}\vspace{-5pt}}
\newcommand{\STAB}[1]{\begin{tabular}{@{}c@{}}#1\end{tabular}}
\newcommand{\bx}{\ensuremath{\mathbf{x}}}
\renewcommand{\vec}[1]{\ensuremath{\mathbf{#1}}}
\newcommand{\matr}[1]{\mathbf{#1}}
\newcommand{\set}[1]{\ensuremath{\mathcal{#1}}}
\newcommand{\SubItem}[1]{
{\setlength\itemindent{15pt} \item[-] #1}
}
\newcommand\ArrowDown[1]{
\hspace{-12px}\rotatebox[origin=c]{270}{$\curvearrowright$}{\hspace{2px}#1}
}
% \vspace{-10mm}
\section{Introduction}
% \vspace{-2mm}
Reconstructing a 3D model from a large set of unordered images is a critical task in computer vision and robotics, with numerous applications in crowd-sourced mapping and other domains.
The most popular approach for 3D reconstruction is the \textit{Structure-from-Motion} algorithm, which estimates both the camera parameters and 3D point cloud simultaneously~\cite{ullman1979interpretation}.
Work in this field generally falls into two categories: \textit{Incremental} \cite{heinly2015reconstructing, schoenberger2016sfm, snavely2006photo, snavely2008modeling, wu2013towards} and \textit{Global methods} \cite{cui2015global, zhu2018largesfm, theia-manual}.
Incremental methods proceed by carefully selecting and adding images one-by-one to the 3D reconstruction while maintaining accuracy through numerical optimization.
Even though such methods lead to particularly accurate results, they can be computationally prohibitive due to the need for frequent bundle adjustment \cite{triggs2000bundle}.
Also, they do not scale well to large image collections.
On the other hand, global methods directly optimize a noisy pose graph and consider all cameras simultaneously, resulting in a faster and more scalable approach.
Although global methods are generally considered less accurate than their incremental counterparts \cite{cui2017hsfm}, they provide a promising research direction for lightweight 3D reconstructions that run in a matter of minutes.
This paper focuses on global algorithms and proposes an algorithm to improve camera rotation estimation by incorporating the gravity direction as additional sensory information into the procedure.
This information is available \textit{by default} in recent consumer devices such as smartphones or mixed reality devices, making it a potentially useful and widely accessible data source.
\begin{figure}[t]
\centering
\includegraphics[width=1.0\columnwidth]{figure/teaser_lamar.png}
\caption{Gravity-aligned rotation averaging.
Given a set of cameras and their gravity directions (left), orientation estimation becomes a 1-degree-of-freedom optimization of the viewing angle (right).
Circular regression tackles the periodicity of the problem.
}
% \vspace{-6mm}
\label{fig:visualization}
\end{figure}
Generally, global methods start with selecting image pairs with a common field-of-view employing image retrieval, such as the visual bag-of-words algorithm \cite{filliat2007visual}.
Then, relative pose estimation is performed between the pairs using a robust estimator, \eg, RANSAC \cite{fischler1981random}.
The resulting relative poses are used to construct an initial pose graph, and camera rotations are estimated simultaneously by \textit{rotation averaging} \cite{govindu2001combining, hartley2013rotation}.
Given the estimated camera rotations, translations and a 3D point cloud are estimated \cite{wilson2014robust}.
Finally, bundle adjustment is performed to refine the 3D reconstruction and camera parameters.
Rotation averaging, also called rotation synchronization, has a rich research literature~\cite{hartley2013rotation, martinec2007robust} and is an integral part of pose graph optimization (PGO) algorithms~\cite{carlone2012linear, wu2023decoupled}.
Typically, it is formulated as a non-linear optimization over the consistency of estimated relative poses and unknown global ones, with the rotation matrices represented by their three degrees of freedom (DoFs).
%
However, modern devices like smartphones, drones and robots, are equipped with inertial measurement units (IMUs), enabling us to pre-estimate the gravity direction accurately~\cite{madgwick2011estimation}.
Such data is widely used in, \eg,
% relative pose estimation~\cite{saurer2015homography,ding2021globally},
autonomous driving \cite{liu2019edge}, unmanned aerial vehicles control \cite{feng2021comprehensive}, and navigation \cite{chang2020gnss, petovello2003real}.
Having a common direction can resolve the yaw and roll of the camera rotation, reducing 2 DoFs in the orientation estimation.
Similar simplification applies to rotation averaging, and consequently, the problem becomes a 1-DoF optimization and aligns with the planar PGO problems \cite{carlone2012linear, carlone2018convex}, as visualized in Fig.~\ref{fig:visualization}.
% Visualization is in Fig.~\ref{fig:visualization}.
Planar pose graph optimization assumes that cameras move in a plane, necessitating only the estimation of the viewing direction.
LAGO~\cite{carlone2014angular}, a representative method in rotation estimation for planar PGO, recognizes that the single degree of freedom permits almost linear estimation, though the periodicity of the variable introduces non-linearity.
To address this, LAGO pre-calculates periods via a heuristic approach and keeps it constant for the subsequent steps.
While many planar PGO methods adopt this approach \cite{carlone2012linear, wu2023decoupled}, the fixed periodicity makes them vulnerable to outliers and error-prone initialization.
% and the failures of the employed heuristic.
The main contribution of this work is a new principled solution to planar rotation averaging.
Unlike prior work~\cite{carlone2012linear, wu2023decoupled} that fix periods and solve linear systems, we tackle the problem via circular regression.
%
Combined with robust losses, it leads to significantly more accurate results on four large-scale and real-world datasets (EuRoC~\cite{burri2016euroc}, KITTI~\cite{geiger2013vision}, LaMAR~\cite{sarlin2022lamar}, and 1DSfM~\cite{wilson2014robust}).
% Our formulation supports efficiently solving the case when only a subset of cameras have known gravity and refining gravity.
Our improvements also extend to the case when only a subset of cameras have known gravity.
We further introduce a refinement method for gravity.
% We will make the source code publicly available.
\begin{figure*}[t]
\centering
\subfloat[EuRoC \cite{burri2016euroc}]{%
\includegraphics[height=0.33\textwidth]{figure/examplary_euroc.png}
\label{fig:examplary_euroc}
}
\subfloat[KITTI \cite{geiger2013vision}]{%
\includegraphics[height=0.33\textwidth]{figure/examplary_kitti.png}
\label{fig:examplary_kitti}
}
\subfloat[LaMAR \cite{sarlin2022lamar}]{%
\includegraphics[height=0.33\textwidth]{figure/examplary_lamar_map.png}
\label{fig:examplary_lamar}
}
\subfloat[1DSfM \cite{wilson2014robust}]{%
\includegraphics[height=0.33\textwidth]{figure/examplary_1dsfm.png}
}
\label{fig:examplary_1dsfm}
\caption{\textbf{Datasets.}
Reconstruction examples from EuRoC~\cite{burri2016euroc}, LaMAR~\cite{sarlin2022lamar}, and 1DSfM~\cite{wilson2014robust}; and trajectories from KITTI~\cite{geiger2013vision}. Our method consistently outperforms baselines across these diverse datasets, showcasing its generalization ability.
}
% \vspace{-6mm}
\label{fig:examplary}
\end{figure*}
% \vspace{-3mm}
\section{Related Work}
% \vspace{-3mm}
% \vspace{1mm}
\noindent\textbf{Global Structure-from-Motion} reconstructs scenes by aggregating pair-wise relative pose estimates.
This is mostly done via rotation averaging~\cite{olsson2011stable,moulon2013global}.
% , but there are also works that perform similarity averaging instead~\cite{cui2015global}.
After determining rotations, translations and structures are recovered.
For instance, Wilson and Snavely~\cite{wilson2014robust} use an outlier filter based on projecting the translations into 1D subspaces.
Moulon et al.~\cite{moulon2013global} employ $L_\infty$-optimization.
Olsson and Enqvist~\cite{olsson2011stable} use $L_\infty$-optimization to solve for 3D points and camera positions.
There are several open-source libraries for global Structure-from-Motion, such as Theia~\cite{theia-manual} and OpenMVG~\cite{moulon2016openmvg}.
We have incorporated our enhanced rotation averaging method into the Theia framework, keeping the rest of the pipeline unchanged.
Nonetheless, our advancements are versatile and not restricted to this particular pipeline.
% \todo{Shorten this section)}\\
% \vspace{1mm}
\noindent\textbf{Rotation Averaging} has a long history in computer vision.
Govindu~\cite{govindu2001combining} linearizes the problem using quaternions, while Martinec and Pajdla~\cite{martinec2007robust} simplify it by omitting certain non-linear constraints on rotation matrices.
%
Wilson et al.~\cite{wilson2016rotations} examine the conditions that make the problem tractable.
Eriksson et al.~\cite{eriksson2018rotation} solve it via strong duality.
Semidefinite programming-based (SDP) relaxation approaches~\cite{arie2012global, fredriksson2012simultaneous} provide optimality guarantees by minimizing the chordal distance~\cite{hartley2013rotation}.
Dellaert et al.~\cite{dellaert2020shonan} sequentially lift the problem into higher-dimensional rotations $\text{SO}(n)$ to avoid local minima where standard numerical optimization techniques may fail~\cite{levenberg1944method,marquardt1963algorithm}.
To handle outliers, various robust loss functions have been investigated \cite{hartley2011l1,chatterjee2013efficient,chatterjee2017robust,sidhartha2021all,zhang2023revisiting}.
Recently, learning-based approaches have emerged.
NeuRoRa \cite{purkait2020neurora}, MSP \cite{yang2021end}, PoGO-Net \cite{li2021pogo} harness Graph Neural Networks to eliminate outliers and regress absolute camera poses.
DMF-synch \cite{tejus2023rotation} employs a matrix factorization technique for pose extraction.
While the above methods offer promising results and runtimes, they operate with 3-DoF rotations, neglecting the advantages of gravity.
% \vspace{1mm}
\noindent\textbf{Leveraging Gravity Direction}
has found its way into numerous applications in computer vision and robotics~\cite{petovello2003real,saurer2015homography,liu2019edge,chang2020gnss,feng2021comprehensive}.
% ,ding2021globally, kukelova2010closed, pautrat2023vanishing}.
Here, we focus on gravity provided by recent consumer devices \textit{by default} through their built-in inertial-based sensory systems \cite{madgwick2011estimation}.
Fraundorfer et al.~\cite{fraundorfer2010minimal} showcase how gravity helps determine yaw and roll in camera rotation, simplifying it to a single degree of freedom.
This simplification extends to rotation averaging, aligning it with planar pose graph optimization techniques.
%Note that these methods can be generalized to any known direction, like the horizontal one.
Planar pose graph optimization was pioneered by Carlone et al.~\cite{carlone2012linear}, designing the LAGO pipeline~\cite{carlone2014fast}.
While being efficient, LAGO is not robust.\footnote{This will be discussed in depth later in Section~\ref{sec:rot_avg_known_dir}, \ref{sec:circ_reg}.}
Subsequent methods proposed different representations: solving in the complex domain~\cite{carlone2016planar, fan2019efficient} and using SDP relaxation~\cite{carlone2018convex}.
Despite their theoretical strengths, they are not robust to outliers in practical applications.
% These methods efficiently handle large-scale data but are not robust to outliers.\footnote{This will be discussed in depth later in Section~\ref{sec:rot_avg_known_dir}, \ref{sec:circ_reg}.}
% Despite their theoretical strengths, their computational demands limit practical applicability.
% We will mainly compare to LAGO \cite{carlone2014fast} as it efficiently handles large-scale data and is widely used in recent practice.
% We also show results from CPL-Sync~\cite{fan2019efficient}, a recent method with theoretical guarantee.
%Our primary comparison is with LAGO \cite{carlone2014fast}, which, while efficient, lacks robustness. In contrast, our proposed method is both versatile and consistently robust across datasets.
%These works lead to an efficient pipeline for solving the planar PGO problem with linear approximation, named LAGO \cite{carlone2014fast}.
%Other works approach the problem with different representations.
%\cite{carlone2016planar} proposes to solve the problem in complex domain, while \cite{carlone2018convex} proposes to use SDP relaxation to solve the problem as 3D case.
%While strong theoretical guarantees are reported for these methods, the inefficiency of these algorithms makes them impractical for real-world applications.
%In this paper, we will mainly compare with LAGO \cite{carlone2014fast}.
%Though LAGO is efficient and generally accurate in dealing with sequential data, it is not robust \cite{carlone2014fast}.
%On the other hand, the proposed algorithm works in general settings and is shown robust across datasets.
% \textit{Gravity Prior in the Rotation Averaging}
Using gravity direction in global SfM has barely been explored.
It has predominantly been viewed as an auxiliary constraint.
Instead of simplifying the problem, it adds complexity to the employed non-linear optimization strategy in prior work.
Both \cite{crandall2012sfm} and \cite{carlone2015initialization} incorporate losses to penalize deviations of the estimated rotational axis from the gravity.
Experimentally, we will demonstrate that directly optimizing 1-DoF rotation can enhance performance and efficiency.
% Experimentally, we will demonstrate that optimizing the rotation directly within a single-parameter domain results in enhanced performance and efficiency.
% \vspace{-2mm}
\section{Gravity-Aligned Rotation Averaging}
% \vspace{-2mm}
\noindent\textbf{Rotation Averaging} is a widely studied problem with several improvements proposed throughout the years.
%
Essentially, the task is to estimate a set of $n \in \mathbb{N}$ rotation matrices $\{\bm{R}_i\} \subset \text{SO}(3)$, given noisy observations of relative rotations $\{\tilde{\bm{R}}_{ij}\} \subset \text{SO}(3)$ estimated in image pairs.
%
The cameras and the relative rotations form a pose graph with the cameras being the vertices $\mathcal{V}$ and relative poses being the edges $\mathcal{E}$.
%
The relationship between the absolute $\bm{R}_i$ and relative $\bm{R}_{ij}$ rotations, for all $(i, j)\in \mathcal{E}$, is as follows:
%
\begin{equation}
\bm{R}_{ij} = \bm{R}_j\bm{R}_i^\top,
\label{eq:r_ij}
\end{equation}
%
where $\bm{R}_{ij}$ represents the noise-free relative rotation between images.
In practice, \eqref{eq:r_ij} never holds due to the noise in the estimations.
Therefore, the problem is typically formulated as a robust least-metric problem as follows:
%
\begin{equation}
\arg\min_{\{\bm{R}_i\}} \sum_{(i,j)\in \mathcal{E}} \rho\left(d(\bm{R}_j^\top \tilde{\bm{R}}_{ij} \bm{R}_i, \bm{I})^p\right),
% := \arg\min_{\{\bm{R}_i\}} E_{\mathcal{E}},
\label{eq:R_err_3dof}
\end{equation}
%
where exponent $p \geq 1$,
$d(\cdot,\cdot)$ is a distance function and
$\rho(\cdot)$ is a robust loss.
A comprehensive discussion on potential distance functions is provided by Hartley \cite{hartley2013rotation}.
%
In the proposed method, we use geodesic error as follows:
%
\begin{equation}
d_{\text{geod}}(\bm{R}_j^\top \tilde{\bm{R}}_{ij} \bm{R}_i, \bm{I}) = \arccos\left(\frac{tr(\bm{R}_j^\top \tilde{\bm{R}}_{ij} \bm{R}_i) - 1}{2}\right),
\label{eq:r_geod}
\end{equation}
%
which leads to a simple formulation.
However, for gradient-based optimization, the chordal distance, defined as the Frobenius norm of the matrix difference
\begin{equation}
d_{\text{chor}}(\bm{R}_j^\top \tilde{\bm{R}}_{ij} \bm{R}_i, \bm{I}) = \|\bm{R}_j^\top \tilde{\bm{R}}_{ij} \bm{R}_i - \bm{I}\|_F,
\label{eq:r_chor}
\end{equation}
%
can also be used as a differentiable objective function.
% \todo{Refine the description of Theia here}
% \vspace{-2mm}
\subsection{Rotation Averaging with Known Direction}
\label{sec:rot_avg_known_dir}
The literature on rotation averaging generally considers the 3-DoF problem, where the cameras are rotated arbitrarily in 3D.
However, recent image-capturing devices are equipped with gyroscopes that provide accurate estimates of the gravity direction in the images.
Such directions can also be extracted from vertical vanishing points \cite{lopez2019deep}, which can also be used in this work.
Having such known directions simplifies the problem, as will be shown in this section.
Let us assume that we are given known direction $\bm{g}_i$ in image $I_i$, which allows for pre-aligning the camera coordinate system with the $y$-axis without loss of generality.
This pre-alignment is done by rotation $\bm{U}_i$ (s.t.\ $\bm{U}_i\cdot [0, 1, 0]^\top \parallel \bm{g}_i$), reducing the degree-of-freedom by 2 and simplifying the camera rotation as follows:
%
\begin{equation}
\bm{R}_i = \bm{U}_i
\begin{bmatrix}
\cos \theta_i & 0 & -\sin \theta_i \\
0 & 1 & 0 \\
\sin \theta_i & 0 & \cos \theta_i
\end{bmatrix}
= \bm{U}_i \bm{R}_{\theta_i},
\label{eq:r_1dof}
\end{equation}
%
where $\theta_i \in [-\pi, \pi)$ is an unknown angle around the $y$-axis.
%
Let us denote the aligned rotations as $\hat{\bm{R}}_i = \bm{U}_i^\top\bm{R}_i= \bm{R}_{\theta_i}$.
Then \eqref{eq:r_ij} can be reformulated as follows:
%
\begin{align}
\bm{R}_{ij}
= \bm{U}_j\hat{\bm{R}}_j\hat{\bm{R}}_i^\top\bm{U}_i^\top
= \bm{U}_j\begin{bmatrix}
\cos (\theta_j - \theta_i) & 0 & -\sin (\theta_j - \theta_i) \\
0 & 1 & 0 \\
\sin (\theta_j - \theta_i) & 0 & \cos (\theta_j - \theta_i)
\end{bmatrix}
\bm{U}_i^\top.
% &= \bm{R}_{\theta_{ij}}
\label{eq:R_ij_1dof}
\end{align}
%
By setting $\hat{\bm{R}}_{ij} = \bm{U}_j^\top\bm{R}_{ij}\bm{U}_i$, we have $\hat{\bm{R}}_{ij} = \bm{R}_{\theta_{ij}}$,
%
%\begin{align}
%\hat{\bm{R}}_{ij} = \bm{R}_{\theta_{ij}},
%\label{eq:R_theta_ij}
%\end{align}
%
where $\theta_{ij} = \theta_j - \theta_i$.
Substituting this formula into \eqref{eq:r_ij}, we get
%
%
\begin{equation}
d_{\text{geod}}(\bm{R}_j^\top \tilde{\bm{R}}_{ij} \bm{R}_i, \bm{I})
= \arccos(\cos[\tilde{\theta}_{ij} - (\theta_j - \theta_i)]) \label{eq:theta_ij}
= [\tilde{\theta}_{ij} - (\theta_j - \theta_i)] + 2k_{ij}\pi,
\nonumber
\end{equation}
%
where $k_{ij} \in \mathbb{Z}$.
By plugging this into the robust least-metric problem, \eqref{eq:R_err_3dof} becomes
%
\begin{equation}
\arg \min_{\{\theta_i\},\{k_{ij}\}\subset\mathbb{Z}} \sum_{(i,j)\in \mathcal{E}} \rho\left(d(\tilde{\theta}_{ij} - (\theta_j - \theta_i) + 2k_{ij}\pi, 0)^p\right).
\label{eq:R_err_1dof_k}
\end{equation}
% Denote $\bar{\epsilon}_{ij} = \tilde{\theta}_{ij} - (\theta_j - \theta_i) + 2k_{ij}\pi$, then $\bar{\epsilon}_{ij} \in [-\pi,\pi)$.
% where $[\tilde{\theta}_{ij} - (\theta_j - \theta_i) + 2k_{ij}\pi] \in [-\pi,\pi)$.
%
Note that this formula is non-linear due to the periodicity of variables $\theta_i$, so standard linear solvers fail.
LAGO \cite{carlone2014fast} addresses this by pre-estimating the periods from a coarse initialization obtained by analyzing the camera trajectory and keeping them fixed afterwards.
% Later, these estimated periods are kept constant.
Such a strategy fails under erroneous initialization, especially in the SfM case that lacks continuous trajectories.
%On the other hand, we leverage circular regression which is robust across datasets.
% our method requires remains robust across datasets.
% \todo{on the contrary, we blablabla}
\input{tex/fig/result_full}
% \vspace{-3mm}
\subsection{Circular Regression}
% \vspace{-1mm}
Circular regression models are proposed to solve systems that contain periodic variables.
Interestingly, this periodic nature of data and its implications have been widely recognized in the field of statistics but not in computer vision. Here, we briefly introduce the problem and refer interested readers to \cite{fisher1992regression} for a more comprehensive discussion.
In general, circular regression falls in the study of circular analysis, which aims to tackle the periodicity of circular data.
Besides the angles in this work, other forms of circular data also widely exist in nature and are studied across fields such as ecology, medical sciences, and cognitive and experimental psychology~\cite{cremers2018one}.
%
Since there is no natural scale measure for circular data, the von Mises distribution (also known as circular normal distribution) is the alternative for the standard Gaussian distribution~\cite{mardia1975statistics}.
% It can be understood as normal distributions wrapped around a specific range, thus it is common to consider circular variables following this distribution.
%
Density function $f$ of a von Mises $\text{VM}(\mu, \kappa)$ distribution is of the following form
%
\begin{equation}
f(\theta; \mu, \kappa) = [2\pi I_0(\kappa)]^{-1} \exp[\kappa \cos(\theta-\mu)],
\end{equation}
%
where $I_0(\kappa)$ is a scaling factor such that
%
% \begin{equation}
$\int_{-\pi}^\pi \exp (\kappa\cos x) dx = 2\pi I_0(\kappa).$
% \end{equation}
%
Parameter $\mu$ is the cluster centers of the distribution while $\kappa$ is a reciprocal measure of dispersion ($\kappa\approx 1/\sigma^2$).
% Von Mises distribution is analogous to normal distributions wrapped around a specific range, and $\kappa$ is analogous to the inverse of variance $1/\sigma^2$.
It is the circular analog of the normal distribution wrapped around a specific range.
When $\kappa \geq 2$, density at $\mu + \pi$ is effectively zero, and it can be well-approximated by a normal distribution.
% In this case, $\kappa\approx\1/\sigma^2$.
Let us consider the problem at hand and assume that each noisy rotation $\tilde{\theta}_{ij}$ is drawn independently and follows a von Mises distribution such that
%
%\begin{equation}
$\tilde{\theta}_{ij} \sim f(\theta; \theta_j - \theta_i, \kappa_{ij})$.
%\end{equation}
%
The probability of the current set of observations $\{\tilde{\theta}_{ij}\}$ is
%
\begin{equation}
\text{Pr}(\{\tilde{\theta}_{ij}\}) = \prod_{(i,j)\in \mathcal{E}} f(\tilde{\theta}_{ij}; \theta_j - \theta_i, \kappa_{ij}).
\label{eq:vm_mle}
\end{equation}
%
The maximum likelihood estimation of \eqref{eq:vm_mle} can be approximated by the following form as in normal distribution:
%
\begin{equation}
\min_{\{\theta_i\}} \sum_{(i,j)\in \mathcal{E}} \kappa_{ij}\left(\tilde{\theta}_{ij} - (\theta_j - \theta_i) + 2k_{ij}\pi\right)^2,
\label{eq:R_err_mle}
\end{equation}
%
where $[\tilde{\theta}_{ij} - (\theta_j - \theta_i) + 2k_{ij}\pi] \in [-\pi,\pi)$.
We assume $\kappa_{ij}$ to be the same for all relative rotations, but they can be leveraged to formulate uncertainty-aware rotation averaging as in \cite{zhang2023revisiting}.
Notice that \eqref{eq:R_err_1dof_k} is a generalized form of the equation, validating our design choice.
Let us note that the proposed formulation follows a similar principle as in Barber's pole model proposed by Gould~\cite{gould1969regression}, and we can similarly apply an iterative procedure to solve the problem.
%
\input{tex/algorithm_circreg}
\label{sec:circ_reg}
The standard procedure minimizing \eqref{eq:R_err_1dof_k} is an iterative optimization.
In each step, the algorithm first decides which period angular variable $\tilde{\theta}_{ij}$ fits in by finding $k_{ij}$ for each term.
Then it solves a linear regression problem, solving linear system \eqref{eq:theta_ij}, with period $k_{ij}$ being fixed.
%
It iterates until convergence, as measured by the change in the estimated variables. Let us denote
\begin{align}
k_{\theta_{i}, \theta_{j}}^* &= \argmin_{k_{ij}\in\mathbb{Z}} |\tilde{\theta}_{ij} - (\theta_j - \theta_i) + 2k_{ij}\pi|, \\
\hat{\epsilon}_{ij} &= \tilde{\theta}_{ij} - (\theta_j - \theta_i) + 2k_{ij}\pi, \quad\hat{\epsilon}_{ij}\in\mathbb{R}.
\label{eq:k_star}
\end{align}
%
The process is described in Alg.~\ref{alg:opt_err_1dof}.
During the iterations, the error in period estimation is gradually rectified.
In contrast, the error in LAGO-like methods~\cite{carlone2014angular,fan2019efficient} persists, resulting in a large performance gap.
See Supp.\ Mat.\ for details.
% Though not guaranteed to converge to global optimal, the algorithm constant achieves accurate result in our experiments.
Note that the algorithm is not guaranteed to converge to global optimal.
However, in practice, we have not found a case where the algorithm stuck at a local minimum with a large error.
For solving the global optimum, the problem can also be formulated as a Mixed Integer Programming, which is NP-Hard \cite{wolsey1999integer} and thus impractical for real-world and large-scale problems.
% Thus, an efficient algorithm which guarantees to find the global
%Thus, while more exact solutions can potentially be found with MIP algorithms, their low efficiency makes them impractical to use.
% \vspace{1mm}
\noindent\textbf{Convergence Properties.}
% %
Denote $d_\text{geod}(\bm{R}_j^\top \tilde{\bm{R}}_{ij} \bm{R}_i, \bm{I})$ as $\bar{\epsilon}_{ij}$, then
by enforcing $\tilde{\theta}_{ij}, \theta_i, \theta_j \in [-\pi, \pi)$, the residual after each iteration is formalized as
%
\begin{equation}
\epsilon_{ij} = \tilde{\theta}_{ij} - (\theta_j - \theta_i) = \bar{\epsilon}_{ij} + 2k_{\theta_i \theta_j}^*\pi ,
\label{eq:residual}
\end{equation}
%
where the number of choices for $k_{\theta_{i}, \theta_{j}}^* \in \{-1, 0, 1\}$ is finite.
We remove the gauge freedom by setting $\theta_1 = 0$, and choose $d(\cdot)$ to be convex.
%
By this, we circumvent the problem of the original Barber pole's model, where we are given an infinite number of peaks (\ie, optimal solutions) with the same probability.
The proposed algorithm inherits the convergence properties of the underlying linear solver.
In each iteration, $k_{ij}$ are set to minimize the respective error.
%
Thus, the change of $k_{ij}$ is guaranteed not to increase $d(\hat{\epsilon}_{ij})$.
Since this holds for all terms, the total error cannot increase.
This means that the overall error after each iteration is monotonically decreasing. Otherwise, the algorithm terminates.
%
Notice that the total number of configurations for $k_{ij}$ is finite, and a configuration cannot be encountered twice due to the strict monotonicity.
Thus, the algorithm will always terminate.
Consequently, the convergence solely depends on the underlying linear solver which has its own strong convergence guarantees.
% The proposed pipeline is based on \cite{chatterjee2013efficient} which is implemented in Theia Library \cite{theia-manual}.
% It proposes a two-stage pipeline that, in the first stage, it minimizes L1 error while in the second stage, it performs IRLS (Iterative Re-weighted Least Squares).
% \vspace{-2mm}
\subsection{Partially Known Gravity}
% \vspace{-0mm}
The assumption that gravity direction is available in all images may be invalid when images are collected from various sources.
Thus, we extend our model for when a subset of images has no known gravity.
Chatterjee et al.~\cite{chatterjee2013efficient} propose an efficient rotation averaging approach based on the axis-angle representation, minimizing the geodesic error.
The distance between two rotations represented by axis-angle $\theta\cdot\bm{v} \in \mathfrak{so}(3)$ approximates the geodesic distance, enabling the linearization of the relative rotation constraint.
We utilize the angle-axis representation for images without known gravity while preserving the single-variable form for those with an established gravity direction.
This categorization results in three types of relative pose constraints:
both images have known gravity;
neither image has gravity;
or one image has gravity while the other lacks it.
The proposed method directly addresses the first scenario.
The second scenario is handled based on \cite{chatterjee2013efficient}.
For the third scenario, we employ a linear system comprised of three constraints, in which the first and third components of $\theta\cdot\bm{v}$ are fixed at zero for images with gravity direction.
All equations are collected together to form a mixed linear system.
We take a stratified approach to solve it.
We first extract the largest connected component for images with gravity and perform the 1-DoF rotation averaging on images within this component.
Afterward, we set the remaining rotation matrices to be identity matrices and solve the mixed linear system.
% Notice that it is also possible to perform rotation averaging per component.
% However, in such a case, we need to solve for the inter-component relative rotation, which requires extra time.
% implement a stratified optimization.
% We start from optimizing only images with gravity, and proceed to consider all images.
Such a design can leverage the fast convergence of 1-DoF images and works well in our experiments.
% \vspace{-3mm}
\subsection{Gravity Direction Refinement}
\label{sec:refinement}
% \vspace{-0mm}
Here, we provide a heuristic approach to improve gravity directions.
In many cases, the inaccuracy of the gravity of a particular image can be recognized by analyzing the set of estimated 3-DoF relative poses between the image and its neighbors.
From Eq.~\eqref{eq:R_ij_1dof}, noisy relative rotation can be pre-aligned as
%
\begin{equation}
\tilde{\bm{R}}_{\theta_{ij}} = \bm{U}_j^\top \tilde{\bm{R}}_{ij} \bm{U}_i.
\label{eq:align_Rij}
\end{equation}
%
Rotations $\tilde{\bm{R}}_{\theta_{ij}}$ provide signal on the validity of $\bm{U}_i$ and $\bm{U}_j$.
Ideally, $\tilde{\bm{R}}_{\theta_{ij}}$ should be a rotation with only 1 DoF.
If the yaw and roll of $\tilde{\bm{R}}_{\theta_{ij}}$ deviate from 0, there are two possibilities: the relative rotation is inaccurate, or at least one of the two gravities is inaccurate.
Based on such an observation, we propose a majority voting mechanism to identify error-prone gravity directions.
For each image, we maintain a counter.
When $\tilde{\bm{R}}_{\theta_{ij}}$ has a large yaw or roll, we increment the counter for both images. % $i$ and $j$.
If the counter for a image is larger than half of its neighbor number, we refine the gravity direction. % marked inaccurate and is refined.
Eq.~\eqref{eq:align_Rij} also allows for refining gravity direction $\bm{g}_{j}$ of the $j$-th image.
%
To estimate $\bm{g}_j$ given $\tilde{\mathbf{R}}_{ij}$ and $\mathbf{U}_i$, we have
% \vspace{-3px}
\begin{equation}
\bm{U}_j \tilde{\bm{R}}_{\theta_{ij}} = \tilde{\bm{R}}_{ij} \bm{U}_i
% \Rightarrow (\bm{U}_j \tilde{\bm{R}}_{\theta_{ij}})_{:,1} = (\tilde{\bm{R}}_{ij} \bm{U}_i)_{:,1}
% \tilde{\bm{R}}_{ij} \bm{U}_i = \bm{U}_j^\top \tilde{\bm{R}}_{\theta_{ij}}\\
\Rightarrow
\bm{g}_{j} = (\bm{U}_j)_{:,1} = (\bm{U}_j \tilde{\bm{R}}_{\theta_{ij}})_{:,1} = (\tilde{\bm{R}}_{ij} \bm{U}_i)_{:,1},
% \vspace{-3px}
\end{equation}
%
where subscript $(\bm{U}_j)_{:,1}$ selects the second column (\ie, down direction) of $\bm{U}_j$.
%
Given multiple neighbors of view $i$, we perform robust estimation of the directions.
We minimize the sum of differences between the estimation and results from neighbors.
To achieve robustness, we optimize the Arctan robust loss using Ceres \cite{ceres_solver}.
% \todo{finish this}
%
%And we can optimize over the gravity estimated from all neighbors to achieve a robust estimation.
Sec.~\ref{sec:ablation_refine} demonstrates the effectiveness of this refinement strategy.
% \subsection{From Hard to Soft Gravity Constraint \label{sec:soft_penalty}}
% \todo{Refine the text here}
% \vspace{1mm}\noindent\textbf{Spectrum of Gravity Constraints}
% For the formulation above, gravity are incorporated to factor out 2 DoF, yet, it is possible to instead incorporated it as an additional penalty term.
% %
% With a similar principle as in \cite{crandall2012sfm}, it penalizes the angle between the rotation axis and the prior.
% %
% The optimization objective then becomes
% \begin{equation}
% \arg\min_{\{\bm{R}_i\}}\eqref{eq:R_err_3dof} + \sum_{i} \lambda \rho\left(d(\theta (\bm{v}_i - (\bm{v}_i^\top\bm{g})\bm{g}))^p\right)
% \label{eq:R_err_penalty}
% \end{equation}
% where $\theta \bm{v}_i \equiv \bm{R}_i$ is the axis-angle representation of $\bm{R}_i$,
% $\bm{g}$ is the gravity, $d$ is the residual, $\rho$ is the robust loss and $\lambda$ controls the level of regularization.
% This formulation allows a continuous spectrum for incorporating gravity constraint.
% %
% When $\lambda = 0$, the formulation restores Eq.\eqref{eq:R_err_3dof}, and when $\lambda = \infty$, the contraint becomes binding, and it can be simplifies to the proposed formulation.
% %
% The analysis details can be found in supp.\ mat.
% %
%------------------------------------------------------------------------
\begin{figure*}[t]
\centering
\includegraphics[width=0.9\textwidth]{figure/legend_less.png}\\
\begin{subfigure}[b]{0.255\textwidth}
\centering
\includegraphics[width=\textwidth]{figure/result_euroc_less.png}
\caption{EuRoC \cite{burri2016euroc}}
\label{fig:result_euroc}
\end{subfigure}
\begin{subfigure}[b]{0.24\textwidth}
\centering
\includegraphics[width=\textwidth]{figure/result_kitti_less.png}
\caption{KITTI \cite{geiger2013vision}}
\label{fig:result_kitti}
\end{subfigure}
\begin{subfigure}[b]{0.24\textwidth}
\centering
\includegraphics[width=\textwidth]{figure/result_lamar_map_ios_less.png}
\caption{LaMAR \cite{sarlin2022lamar}}
\label{fig:result_lamar}
\end{subfigure}
\begin{subfigure}[b]{0.24\textwidth}
\centering
\includegraphics[width=\textwidth]{figure/result_1dsfm_less.png}
\caption{1DSfM \cite{wilson2014robust}}
\label{fig:result_1dsfm}
\end{subfigure}
\caption{
The cumulative distribution functions (CDFs) of the absolute rotation errors ($^\circ$). Estimated by the Levenberg-Marquardt~\cite{more2006levenberg} method solving the 3-DoF (LM 3DoF) and 1-DoF (LM 1DoF) problems, by LAGO~\cite{carlone2014fast}, by CPL-Sync~\cite{fan2019efficient}, by the rotation averaging in the Theia library~\cite{theia-manual,chatterjee2013efficient}, by Theia with an additional penalty term (Theia$^\text{reg}$ \cite{crandall2012sfm}), by COLMAP~\cite{schoenberger2016sfm}, and by the proposed method.
Curve "Gravity" stands for the approximate upper bound achievable by using gravity direction.
A method being accurate is interpreted by its curve close to the top-left corner.
%
Different metrics for the datasets are reported in Tables~\ref{tbl:euroc}, \ref{tbl:KITTI}, \ref{tbl:lamar_map}, \ref{tbl:1dsfm}. }
\label{fig:result_full}
% \vspace{-4mm}
% \vspace{-10px}
\end{figure*}
%\begin{algorithm}[t]
%\caption{Solve optimization for equation \eqref{eq:R_err_1dof_k}}
%\label{alg:opt_err_1dof}
%\begin{algorithmic}[1] % The number tells where the line numbering should start
%\State $\theta_i \gets 0$ \Comment{Initialize $\theta_i$ if not already initialized}
%\State $k_{ij} \gets k_{\theta_{i}, \theta_{j}}^*$
%\While{not converged \textbf{and} $ite < max\_ites$}
% \State $\theta_i \gets \argmin_{\{\theta_i\}} \sum_{(i,j)\in \mathcal{E}} \rho\left(d(\hat{\epsilon}_{ij})^p\right)$
% \State $k_{ij} \gets k_{\theta_{i}, \theta_{j}}^*$
%\EndWhile
%\end{algorithmic}
%\end{algorithm}
\begin{figure}[t]
\begin{center}
\scalebox{0.85}{
\begin{minipage}{\textwidth}
\vspace{0pt}
\begin{algorithm}[H] \small
\caption{Solve optimization for \eqref{eq:R_err_1dof_k}}
\label{alg:opt_err_1dof}
\DontPrintSemicolon
\SetAlgoLined
\SetNoFillComment
$\theta_i\gets 0$ if $\theta$ \text{not initialized}\;
$k_{ij} \gets k_{\theta_{i}, \theta_{j}}^*$ \;
\While{not converged \textbf{and} $ite < max\_ites $} {
$\theta_i \gets \argmin_{\{\theta_i\} } \sum_{(i,j)\in \mathcal{E}} \rho\left(d(\hat{\epsilon}_{ij})^p\right)$ \tcp*{optimize $\theta_i$ with fixed $k_{ij}$}%\;
$k_{ij} \gets k_{\theta_{i}, \theta_{j}}^*$ \tcp*{optimize $k_{ij}$ with fixed $\theta_i$}%\;
}
\end{algorithm}
\end{minipage}
}
\end{center}
\end{figure}
% \vspace{-4mm}
% \vspace{-2mm}
\section{Experiments}
% \vspace{-2mm}
% \todo{UPDATE CONCRETE NUMBERS IN THE TEXT}\\
% \todo{Put the citation in the text}
% \todo{Add the comparison with the baseline model (soft penalty on the deviation from the gravity prior)}
% In the experiments, we mainly compared with Theia library \cite{theia-manual} implementation of \cite{chatterjee2013efficient} .
%
% \vspace{1mm}
\noindent\textbf{Implementation.}
%
We implemented the proposed rotation averaging in C++ in a similar fashion as the widely used Theia library \cite{theia-manual}.
%
% The code first optimizes the $L_1$-norm on the estimated relative poses.
% Second, it performs the proposed iteratively re-weighted least square (IRLS) problem.
% \vspace{1mm}
\noindent\textbf{Distance Functions and Robust Loss.}
Having an ordinary linear system in each step of the algorithm allows us to use any $n$-norm as a distance function and robust loss, such as Huber \cite{huber1992robust} or Cauchy Loss \cite{black1996robust}.
%
To be robust to outliers, we follow the two-stage optimization strategy of Theia \cite{theia-manual}.
In the first stage, we minimize $L_1$ distance.
In the second one, we use an iteratively re-weighted approach.
The weights are determined by the Geman-McClure function \cite{ganan1985bayesian}.
% This loss has a bell-shaped curve that converges to $0$ as the residual grows, which makes it robust to outliers.
The parameter choice follows the Theia library~\cite{theia-manual}.
% \vspace{1mm}
\noindent\textbf{Competitors.}
%
We compare the proposed rotation averaging to
LAGO \cite{carlone2014fast}, implemented in GTSAM library \cite{gtsam}, to CPL-Sync \cite{fan2019efficient},
with \cite{chatterjee2013efficient} and \cite{crandall2012sfm} implemented in the Theia library~\cite{theia-manual},
and with methods optimizing the 3-DoF and 1-DoF problems by the Levenberg–Marquardt (LM) algorithm \cite{levenberg1944method} with the Ceres library \cite{ceres_solver}.
CPL-Sync~\cite{fan2019efficient} is a planar PGO algorithm.
For the LM optimization, we use the Soft-$L_1$ robust loss function \cite{ceres_solver}.
To be differentiable and exploit the periodicity, we use the scaled $\sin$ as distance function as
%
%\begin{equation}
$d_{\text{1-DoF}}(\epsilon) = 2\sqrt{2}\sin (\epsilon/2)$.
%\end{equation}
%
Note that the error is equivalent to the chordal error \cite{hartley2013rotation}.
We use the chordal error in the 3-DoF optimization for a fair comparison.
Additionally, we compare with another baseline.
Similarly as in \cite{crandall2012sfm}, we add an additional error term penalizing the angle between the rotation axis and the gravity direction.
The objective function becomes:
\begin{equation}
\arg\min_{\{\bm{R}_i\}}E_{\mathcal{E}} + \sum_{i} \lambda \rho\left(d(\theta \cdot \left(\bm{v}_i - (\bm{v}_i^\top\bm{g}_i)\bm{g}_i)\right)^p\right),
% \arg\min_{\{\bm{R}_i\}}\eqref{eq:R_err_3dof} + \sum_{i} \lambda \rho\left(d(\theta \cdot \left(\bm{v}_i - (\bm{v}_i^\top\bm{g}_i)\bm{g}_i)\right)^p\right),
\label{eq:R_err_penalty}
\end{equation}
where $E_{\mathcal{E}}$ is the same as Eq. \eqref{eq:R_err_3dof}, $\theta\cdot\bm{v}_i \equiv \bm{R}_i$ is the axis-angle representation of $\bm{R}_i$,
$\bm{g}_i$ is the gravity, $d$ is the distance function, $\rho$ is the robust loss and $\lambda$ controls the level of regularization.
In the experiments, we set $\lambda$ to 1 as in \cite{carlone2015initialization} and denote this method as \textit{Theia$^\text{reg}$}.
See implementation details in supp. mat.
% For the ease of implementation, we pre-align the camera rotations with the gravity prior.
% Thus, the estimated rotations are around $(0, 1, 0)^\top$.
% In this case, the first and the last term for camera rotations should be 0.
% We can directly penalize these two terms as the deviation from the prior.
% After appending these linear systems to the system, the problem can be solved as in Theia~\cite{theia-manual}.
% In implementation, we
% To achieve such implementation, we
In practice, it is possible to modify the Theia~\cite{theia-manual} pipeline to only update the rotation around $y$-axis.
While the objective function of the proposed method and a modified Theia formulation are the same, the latter solves it through a more complex procedure.
As a result, the running time of such a formulation is several times slower than the proposed method.
Also, it is not applicable when the gravity is partially known.
See details in supp. mat.
% Detailed derivation and results can be found in the Supplementary Material.
We also tested recent learning-based alternatives \cite{purkait2020neurora, yang2021end, li2021pogo, tejus2023rotation}.
While they report promising results on the 1DSfM datasets~\cite{wilson2014robust}, they fall short on the tested sequential ones \cite{burri2016euroc, geiger2013vision, sarlin2022lamar}.
This is likely caused by the connectivity of the pose graph being too sparse.
We included their results in the Supplementary Material.
\input{tex/tbl/euroc}
While we compare to global SfM algorithms, we also show the results of COLMAP \cite{schoenberger2016sfm} (\ie, incremental SfM).
Even though COLMAP produces high-quality reconstructions, it runs significantly slower than global SfM in practice.
% \vspace{1mm}
\noindent\textbf{Comparison.}
%
In all tests, the relative poses are estimated by LO-RANSAC \cite{chum2003locally}, obtaining the 5-DoF essential matrix \textbf{E}.
Experiments with 3-DoF \textbf{E} are included in the supp. mat.
In short, it improves the results for both our method and Theia~\cite{theia-manual} but the relative performance remains the same.
% In short, it can decrease the gap between our method and 3 DoF algorithms but
% Let us note that using gravity-aligned \textbf{E} \cite{sweeney2014solving} did not improve in our experiments.
To address the gauge ambiguity in comparing with the ground truth (GT), we perform a global 3-DoF rotation alignment by minimizing the Cauchy Loss using Ceres~\cite{ceres_solver}.
Geodesic error $d_{\text{geod}}$ between estimated $\bm{R}_i$ and GT $\bm{R}^{gt}_i$ is considered as rotation error.
% The rotation error, given estimated $\bar{\bm{R}}_i$ and GT $\bm{R}_i$, is calculated as the geodesic error $d_{\text{geod}}$.
% \vspace{-2mm}
\subsection{Rotation Averaging with Measured Gravity}
% \vspace{-1mm}
% ~\\
\noindent\textbf{EuRoC \cite{burri2016euroc}} is a visual-inertial dataset collected by a Micro Aerial Vehicle (Figure \ref{fig:examplary_euroc}).
The dataset contains stereo images and synchronized IMU measurements and is millimeter-accurate.
We align the gravity to the first frame and use \cite{madgwick2010efficient} from \cite{madgwick_filter} to calculate gravity direction from the raw measurements.
% Example images and reconstruction can be found in Figure \ref{fig:examplary_euroc}.
On average, the sequences comprise 1077 images and 12620 pairwise motion estimates.
The average and median rotation errors in degrees, the Area Under the recall Curve (AUC) thresholded at $0.5^\circ$, $1^\circ$, and $2^\circ$, and the average runtime (in hours) are reported in Table \ref{tbl:euroc}.
The 1-DoF optimization performs significantly more accurately and faster than the 3-DoF optimization, especially when the gravity is accurate, motivating its use.
%
Also, the method implemented in the Theia library is significantly more advanced than a simple LM optimization, achieving better accuracy than the 3-DoF and 1-DoF optimizations.
Compared to LAGO and CPL-Sync, the proposed method achieves much higher accuracy and is more efficient.
The same holds compared to Theia \cite{theia-manual}.
% When compared to Theia with extra penalty term (Theia$^\text{reg}$), we achieve better results in terms of median error, AUC at $0.5^\circ$ and $1^\circ$, and we are comparable in average error and AUC at $2^\circ$.
When compared to Theia with an extra penalty term (Theia$^\text{reg}$), we achieve better or comparable results in all metrics while being an order of magnitude faster.
% Also, we are an order of magnitude faster.
The proposed method is substantially faster and more accurate than COLMAP in all metrics.
The cumulative distribution functions (CDF) of the absolute rotation errors are in Fig. \ref{fig:result_euroc}.
A method being accurate is interpreted as its curve close to the top-left corner.
Curve ``Gravity'' (dashed curve) shows the accuracy of gravity bound, and is in principle the upper bound of the gravity-based methods.
%
It is calculated as the geodesic error between the GT rotation and the closest rotation matrix, which aligns with the current gravity direction.
Precisely, the error is
%
%\begin{equation}
$\epsilon_{\text{grav}, i} = d_{\text{geod}}(\bm{R}_i^\top\bm{\tilde{U}}_{i}\bm{R}_{\theta_{i}}, \bm{I})$,
%\label{eq:err_grav}
%\end{equation}
%
where $\bm{\tilde{U}}_i$ aligns the noisy gravity.
% Notice that compared with all the baselines except for Theia with fixed axis (olive dotted line), we are closer to upper left corner, indicates that the
% Our method performs similarly as Theia$^\text{1DoF}$ while outperforming other baseline models evidently.
Our method closely approaches the gravity upper bound curve.%, indicating its capability.
% Also, it substantially outperforms Theia, LAGO
% The cumulative distribution functions (CDF) of the relative rotation errors are shown in Fig. \ref{fig:result_euroc}.
% Curve "Input" (gray dash-dotted curve) is the error of the input relative poses estimated by LO-RANSAC and optimized by bundle adjustment on each image pair independently.
% The proposed algorithm is the closest to the upper left corner compared with other methods.
% We observe similar behavior as on the LaMAR dataset.
% The direct 3-DoF optimization is slow and leads to inaccurate results.
% The direct 1-DoF method achieves accurate rotations compared to the 3-DoF problem and is the fastest algorithm on this dataset.
% The AUC scores of Theia are significantly worse than that of LM 1-DoF while also being slower.
% The proposed rotation averaging algorithm achieves the highest accuracy, again, by a significant margin.
% The Theia with fixed axis perf
% Compared with Theia with a fixed rotation axis and additional error term, the proposed method has similar accuracy, but is more than 3 times faster.
% \subsection{Rotation Averaging with Gravity Prior}
% In this experiment, we analyze the effect of using a rough gravity prior, namely, assuming that it is $[0, 1, 0]^\top$, \ie, it points downwards.
% While this assumption might be inaccurate for optimization in a dataset exhibiting general camera movements, it is a usual assumption for the autonomous driving scenario.
% % In this case, the camera is mounted to a moving vehicle and, thus, it is approximately aligned with gravity throughout the entire trajectory.
% In this case, the camera is mounted to a moving vehicle, so if the gravity direction of it is pre-calibrated, it approximately stay aligned with gravity throughout the entire trajectory.
\input{tex/tbl/KITTI}
% \vspace{1mm}
\noindent\textbf{KITTI \cite{geiger2013vision}} is a real-world autonomoust driving visual odometry benchmark with GT from GPS localization system (Fig.~\ref{fig:examplary_kitti})
% from the laser scanner and GPS localization system (Fig.~\ref{fig:examplary_kitti}).
% With the city being mostly flat, we adopt the assumption that gravity is pointing downward in the image. We tested our method on the 11 visual odometry sequences that are provided with ground truth.
%
We tested our method on the 9 odometry sequences (without 00, 03) provided with ground truth.
For sequence 00, several data intervals are missing, and there is no raw IMU data for sequence 03.
% Visualization is in Fig. \ref{fig:examplary_kitti}.
In the tested sequences, there are 1984 images and we form 20767 image pairs on average.
% the average number of images is 1984, and the average number of pairwise motion estimates is 20767.
We use the Invariant Extended Kalman Filter implemented in \cite{brossard2020ai} to estimate the gravity direction from raw synchronized IMU data.
% The training epoch is set to 0 and it collapse to an Invariant Extended Kalman Filter.
The results are in Table \ref{tbl:KITTI}.
The proposed method outperforms Theia~\cite{theia-manual} by a large margin, featuring 23, 22, and 12 points improvement in AUC score at $0.5^\circ$, $1^\circ$ and $2^\circ$ respectively, and is about 8 times faster.
Compared to Theia$^\text{reg}$ \cite{crandall2012sfm}, the proposed method achieves substantial accuracy improvements while running 30 times faster.
% The performance of the proposed method and Theia$^\text{1DoF}$ is similar, with the former running 4 times faster.
%taking only 1/4 of the running time.
% As for LAGO~\cite{carlone2014fast}, the proposed method achieves a much higher accuracy at the cost of shorter running time.
%Similar trends hold for LAGO~\cite{carlone2014fast}.
Even though LAGO and CPL-Sync are designed for this scenario, they perform inaccurately due to the missing of a robust objective function.
% We attribute this to the sensitive period pre-estimation (more results are in the Supp.\ Mat.).
% We attribute this to the robustness of the two-stage optimization.
% The median error of the proposed method is about 1/10 and the AUC score at $0.5^\circ$ is about 2 times of LAGO.
Compared to COLMAP, the estimation is significantly more accurate and efficient.
Due to the estimation drift, COLMAP~\cite{schoenberger2016sfm} accumulates large errors along the sequence, while the global methods generally achieve low estimation error.
% COLMAP fails for a large portion of the data, majorly due to the estimation drift, while the global methods generally achieve low estimation error.
% \todo{Double check the result}
% One can notice that the
The cumulative distribution functions (CDF) of the absolute rotation errors are shown in Fig.~\ref{fig:result_kitti}.
%
The proposed method (red) is the closest to the top-left corner.
It approaches the gravity upper bound closely and is significantly more accurate than the other compared algorithms.
%
% The relative pose error in the bottom row shows similar trends with the proposed method being the best.
% Theia, Theia$^\text{reg}$ and Theia$^\text{1DoF}$ and our method shows comparable results for relative pose error.
\input{tex/tbl/slam_compare}
Table \ref{tbl:slam_compare} compares the proposed method with SLAM systems.
The SLAM results (provided on 5 sequences) are copied from \cite{sharafutdinov2023comparison}.
% Only available datasets are listed.
The proposed algorithm is the most accurate on EuRoC by a large margin.
It is the best on two sequences of KITTI, and, on the third, it is on par with the best one with only $0.12^\circ$ difference.
%
Note that several SLAM pipelines also take IMU data into consideration and, potentially, other sensors, such as VINS-Fusion~\cite{qin2019general} that uses stereo cameras.
% Methods such as VINS-Fusion~\cite{qin2019general} are with additional sensors like stereo-cameras.
These results clearly demonstrate the robustness of the proposed method.
% \vspace{-2mm}
\subsection{Experiments with Partially Known Gravity}
\noindent\textbf{LaMAR \cite{sarlin2022lamar}} is a recent large-scale benchmark.
Example images and reconstruction can be found in Figure \ref{fig:examplary_lamar}.
The dataset contains images captured by Hololens 2 \cite{hololens} and smartphones.
Smartphone images are processed with ARKit~\cite{arkit}, estimating gravity-aligned trajectory via onboard inertial units.
Such estimations are available \textit{by default}, and can be readily used in applications.
We use gravity information captured by smartphones and all sequences from three scenes.
The considered sequences consist of 31676 images with 194182 image pairs on average, and 25\% of images are with gravity directions.
Since we focus on the partial gravity case here, we do not consider the gravity from the Hololens devices.
Additional results are in the Supplementary Material.
%
% It runs in real-time on the LaMAR dataset.
\input{tex/tbl/lamar_map}
Results are in Table \ref{tbl:lamar_map}.
We do not show results for LAGO~\cite{carlone2014angular} and CPL-Sync~\cite{carlone2014fast} as they are not applicable in this scenario.
The proposed method outperforms Theia-based methods in all metrics while running 1.5 times faster.
It is on par with COLMAP, having a marginally lower AUC $0.5^\circ$ score and being better in other metrics.
However, COLMAP runs four orders of magnitude slower, in a matter of hours instead of seconds.
The cumulative distribution functions (CDF) of the absolute rotation errors are shown in Fig. \ref{fig:result_lamar}.
The proposed method (red) is the closest to the top-left corner, showing that it leads to the lowest absolute errors.
% \vspace{-4mm}
\subsection{Semi-Synthetic Experiments}
%In this section, we evaluate the proposed algorithm on a typical Structure-from-Motion dataset.
% \vspace{1mm}
\noindent\textbf{1DSfM \cite{wilson2014robust}} contains Internet photo collections, featuring the Structure-from-Motion task on unordered image sets.
The gravity direction for each image is not provided. Thus, we use the ground truth direction and add zero-mean Gaussian noise.
We set the standard deviation of the gravity error to $0.5^\circ$. % on gravity by applying random rotations.
This is slightly larger than the noise levels observed in the EuRoC, KITTI, and LaMAR datasets.
We consider the reconstruction in the original dataset as ground truth and use the relative poses provided as input.
\input{tex/tbl/1dsfm}
The results are summarized in Table \ref{tbl:1dsfm}.
For all methods, we used improved gravities as input.
We see similar trends to those of the other datasets.
% Results here
The direct 1-DoF optimization improves significantly upon the 3-DoF method.
The proposed algorithm leads to the lowest median error and the highest AUC scores.
It is about 15 times faster than the Theia-based methods.
Compared with LAGO~\cite{carlone2014fast} and CPL-Sync~\cite{fan2019efficient}, the proposed method obtains significantly more accurate results.
% With a
% With the noisy relative poses as input, the
% The relative pose in this datasets contains higher ratio of outliers.
%
% The AUC scores for the proposed method are also higher than those of the other methods.
For this dataset, we do not report COLMAP results as the ground truth is from another structure-from-motion method. Thus, the error is not directly comparable.
%
The cumulative distribution functions (CDF) of the absolute rotation errors are shown in Fig. \ref{fig:result_1dsfm}.
The proposed method (red) is the closest to the top-left corner, and it clearly outperforms other baselines.
\noindent
\textbf{Ablation Study for Gravity Refinement.\label{sec:ablation_refine}}
%
Table \ref{tbl:refinement} shows the refinement performance on the 1DSfM~\cite{wilson2014robust} dataset.
Even with noisy relative pose observations, rotation averaging largely benefits from performing gravity direction refinement.
The improvement is especially significant for the AUC scores at $0.5^\circ$ and $1^\circ$.
The runtime overhead is less than $10\%$.
Note that we observed no improvements on the sequence-based datasets.
We attribute this to the fact that the gravity noise is not independent in such cases, \eg, it stems from drifting.
\input{tex/tbl/refine}
% \vspace{-2mm}
\subsection{Synthetic Experiments}
To show the scalability of the gravity-aligned optimization, we conduct tests by generating graphs with a varying number of edges and cameras.
We experiment with two types of view graphs, \ie, sequentially and grid-connected ones, to mimic the image stream and unordered image sets settings.
For the sequential setting, each camera is connected to its 20 nearest neighbors, which leads to graphs with similar connectivity as the KITTI and EuRoC datasets.
For grid data, each camera is connected with its 24 closest neighbors to have a similar number of neighbors to the sequential case.
The results are summarized in Figure \ref{fig:runtime}.
For all methods except for LAGO and CPL-Sync, the runtime depends linearly on the number of images in the graph.
Note that the edge number is a constant times the image number.
The runtime of LAGO undergoes a sudden increase after the images exceed 25600, and the implementation \cite{gtsam} crashes unexpectedly when the number of cameras reaches 102400 for the sequential case.
The proposed method is an order of magnitude faster than Theia-based baselines and the 3-DoF optimization.
% \todo{compare to others}
%
The reason is two-fold: we achieve a faster convergence while solving a problem with fewer parameters.
Both lead to a more efficient and stable algorithm.
The proposed method is several times faster than LAGO~\cite{carlone2014angular}, mainly due to pipeline design.
% \subsection{Adaptive Loss Parameter} \label{sec:sigma}
% To show the effectiveness of our $\sigma$ selection, we compare the AUC@$5^\circ$ score with different $\sigma$ on a randomly chosen sequence from each dataset.
% Results are in Figure~\ref{fig:sigma}.
% Across different datasets, the selected $\sigma^*$ shows large variance while consistently yielding close-to-optimal accuracy, indicating the effectiveness of the proposed method.
% \begin{figure}
% \centering
% \includegraphics[width=\columnwidth]{figure/sigma.png}
% \caption{The AUC@$5^\circ$ score (vertical axis) of the proposed algorithm (left) and Theia~\cite{theia-manual} plotted as a function of the loss parameter $\sigma$ (horizontal; Section~\ref{sec:sigma_setting}), and $\sigma^*$, shown by stars, set adaptively by Eq.~\ref{eq:sigma} on each dataset.
% \todo{Update the sigma figure here (with the new mechanism }
% }
% \label{fig:sigma}
% \end{figure}
\begin{figure}[t]
\centering
\includegraphics[width=0.7\columnwidth]{figure/runtime_less.png}
\caption{
Avg.\ $\log_{10}$ time (secs) as a function of image number in synthetic pose graphs: sequential (left) and grid-like (right) cameras.
% Missing data points are due to crashed or unterminated programs.
% placement.
%run time (sequential neighbors), runtime (grid neighbors)
}
% \vspace{-2mm}
\label{fig:runtime}
\end{figure}
%However, we observed that when the gravity noise is not independent, for instance, when the errors come from measurement drifting, refinement does not improve.
%Thus, we observed no improvements on the sequence-based datasets.
%However, if the prior error is not independent, for instance, when errors come from measurement drifting
% , more delicately designed mechanism is required to rectify the error.
% \todo{mention at the dataset where we improve gravities, and that we used he improved gravites in all method}
\begin{table}[t]
\centering
\caption{Average and median rotation errors ($^\circ$), the AUC score at $0.5^\circ$, $1^\circ$, and $2^\circ$ and average run-time (hour) on EuRoC~\cite{burri2016euroc}.}
% \vspace{-2mm}
\resizebox{0.75\columnwidth}{!}{
\begin{tabular}{l c c c c c c} \toprule
& AVG ($^\circ$) & MED ($^\circ$) & AUC@$0.5^\circ$ &~~~~@$1^\circ~~~~$ &~~~~@$2^\circ~~~~$ & Time (h) \\ \midrule
LM 3DoF & 3.93 & 0.96 & 11.13 & 23.38 & 41.33 & 2.94 $\times 10^{-4}$ \\
LM 1DoF & 0.74 & 0.43 & 27.55 & 46.02 & 66.47 & \textbf{1.11 $\times 10^{-5}$} \\
LAGO \cite{carlone2014fast} & 0.83 & 0.45 & 27.18 & 45.41 & 64.41 & 8.61 $\times 10^{-5}$ \\
CPL-Sync \cite{fan2019efficient} & 0.82 & 0.45 & 26.75 & 44.82 & 63.64 & 2.67 $\times 10^{-4}$ \\
Theia \cite{theia-manual} & 0.82 & 0.53 & 22.14 & 43.85 & 66.06 & 4.25 $\times 10^{-4}$ \\
Theia$^\text{reg}$ \cite{crandall2012sfm} & \textbf{0.63} & 0.40 & 29.20 & 52.86 & 72.72 & 5.83 $\times 10^{-4}$ \\
Proposed & 0.64 & \textbf{0.35} & \textbf{33.06} & \textbf{54.81} & \textbf{72.97} & 5.56 $\times 10^{-5}$ \\
\hdashline
COLMAP \cite{schoenberger2016sfm} & 1.17 & 0.77 & 12.93 & 32.27 & 55.82 & 5.05 \\
\bottomrule
\end{tabular}}
% \vspace{-4mm}
\label{tbl:euroc}
\end{table}
\begin{table}[t]
\centering
\caption{Average and median rotation errors ($^\circ$), the AUC score at $0.5^\circ$, $1^\circ$, and $2^\circ$ and average run-time (hour) on KITTI~\cite{geiger2013vision}.
}
% \vspace{-3mm}
\resizebox{0.75\columnwidth}{!}{
\begin{tabular}{l c c c c c c} \toprule
& AVG ($^\circ$) & MED ($^\circ$) & AUC@$0.5^\circ$ &~~~~@$1^\circ~~~~$ &~~~~@$2^\circ~~~~$ & Time (h) \\ \midrule
LM 3DoF & 3.14 & 2.73 & 15.60 & 24.80 & 35.29 & 5.28 $\times 10^{-4}$ \\
LM 1DoF & 2.77 & 2.36 & 24.42 & 32.84 & 41.88 & \textbf{1.67 $\times 10^{-5}$} \\
LAGO \cite{carlone2014fast} & 5.50 & 3.17 & 19.14 & 27.15 & 37.38 & 8.06 $\times 10^{-5}$ \\
CPL-Sync \cite{fan2019efficient} & 4.38 & 2.95 & 23.26 & 31.08 & 40.17 & 5.00 $\times 10^{-5}$ \\
Theia \cite{theia-manual} & 0.68 & 0.64 & 25.96 & 47.48 & 71.77 & 3.92 $\times 10^{-4}$ \\
Theia$^\text{reg}$ \cite{crandall2012sfm} & 0.54 & 0.31 & 45.38 & 64.85 & 78.83 & 1.56 $\times 10^{-3}$ \\
% Theia$^\text{1DoF}$ & \textbf{0.39} & \textbf{0.29} & 48.70 & \textbf{69.79} & \textbf{83.86} & 2.06 $\times 10^{-4}$ \\
Proposed & \textbf{0.39} & \textbf{0.29} & \textbf{48.71} & \textbf{69.79} & \textbf{83.86} & 5.56 $\times 10^{-5}$ \\
\hdashline
COLMAP \cite{schoenberger2016sfm} & 3.76 & 3.13 & 20.09 & 28.29 & 36.91 & 8.80 \\
\bottomrule
\end{tabular}}
% \vspace{-3mm}
\label{tbl:KITTI}
\end{table}
\begin{table}[t]
\centering
\caption{Comparison with SLAM-based method. Root-mean-square rotation error ($^\circ$) are reported.
% Only available SLAM results are copied from \cite{sharafutdinov2023comparison}.
The SLAM results (only 5 are available) are copied from \cite{sharafutdinov2023comparison}
% and only these 5 entries available.
}
% \vspace{-3mm}
\resizebox{\columnwidth}{!}{
\begin{tabular}{lrccccccc}
\toprule
% \multicolumn{2}{c}{\multirow{2}{*}{Dataset}} & ORB-SLAM2 & LDSO & VINS-Fusion & OpenVSLAM & Basalt & ORB-SLAM3 & \multirow{2}{*}{Proposed} \\
% & & \cite{mur2017orb} & \cite{gao2018ldso} & \cite{qin2019general} & \cite{sumikura2019openvslam} & \cite{usenko2019visual} & \cite{campos2021orb} & \\
\multicolumn{2}{c}{Dataset} & ORB-SLAM2\cite{mur2017orb} & LDSO\cite{gao2018ldso} & VINS-Fusion\cite{qin2019general} & OpenVSLAM\cite{sumikura2019openvslam} & Basalt\cite{usenko2019visual} & ORB-SLAM3\cite{campos2021orb} & Proposed \\
% & & \cite{mur2017orb} & \cite{gao2018ldso} & \cite{qin2019general} & \cite{sumikura2019openvslam} & \cite{usenko2019visual} & \cite{campos2021orb} & \\
\midrule
\multirow{2}{*}{EuRoC} & V1\_01 & 4.70 & \phantom{1}4.68 & 5.60 & 4.73 & 5.36 & 5.94 & \textbf{2.16} \\
& MH\_05 & 6.60 & 20.45 & 1.87 & 6.54 & 0.67 & 0.93 & \textbf{0.21} \\
\cmidrule{1-9}
\multirow{3}{*}{KITTI} & 02 & 1.47 & \phantom{1}2.41 & 4.78 & 1.20 & 1.72 & 1.43 & \textbf{0.62} \\
& 05 & 0.38 & \phantom{1}0.87 & 3.02 & 0.38 & 0.86 & \textbf{0.37} & 0.49 \\
& 06 & 0.39 & \phantom{1}0.59 & 4.81 & 0.69 & 2.78 & 0.41 & \textbf{0.25} \\
\bottomrule
\end{tabular}}
% \vspace{-2mm}
\label{tbl:slam_compare}
\end{table}
\begin{table}[t]
\centering
\caption{Average and median rotation errors ($^\circ$), AUC score at $0.5^\circ$, $1^\circ$, and $2^\circ$ and average run-time (hours) on LaMAR~\cite{sarlin2022lamar}. %\cite{carlone2014fast, fan2019efficient} are not applicable when gravity is partially known.
}
% \vspace{-3mm}
\resizebox{0.75\columnwidth}{!}{
\begin{tabular}{l c c c c c c} \toprule
& AVG ($^\circ$) & MED ($^\circ$) & AUC@$0.5^\circ$ &~~~~@$1^\circ~~~~$ &~~~~@$2^\circ~~~~$ & Time (h) \\ \midrule
% LM 3DoF & 52.30 & 32.36 & 0.07 & \phantom{a}0.35 & \phantom{a}1.26 & 4.08 $\times 10^{-2}$ \\
% LM 1DoF & 44.90 & 31.61 & 0.28 & \phantom{a}0.73 & \phantom{a}1.81 & 1.69 $\times 10^{-2}$ \\
% Theia \cite{theia-manual} & \phantom{a}\textbf{3.46} & \phantom{a}1.28 & 3.28 & 13.92 & 34.62 & 3.88 $\times 10^{-2}$ \\
% % Theia$^+$ \cite{crandall2012sfm} & \phantom{a}6.30 & \phantom{a}1.12 & 6.11 & 18.89 & 39.79 & 3.72 $\times 10^{-2}$ \\
% % Theia$^\infty$ & 10.75 & \phantom{a}2.02 & 6.30 & 15.14 & 28.04 & 3.74 $\times 10^{-2}$ \\
% Theia$^+$ \cite{crandall2012sfm} & \phantom{a}8.14 & \phantom{a}1.13 & 5.38 & 17.75 & 39.55 & 3.68 $\times 10^{-2}$ \\
% Theia$^\infty$ & \phantom{a}8.77 & \phantom{a}1.72 & 7.21 & 17.06 & 31.07 & 5.16 $\times 10^{-2}$ \\
% Proposed & \phantom{a}7.19 & \phantom{a}\textbf{1.06} & 7.63 & 20.46 & \textbf{42.02} & \textbf{1.21 $\times 10^{-2}$} \\
% \hdashline
LM 3DoF & 51.79 & 35.53 & 0.06 & \phantom{a}0.30 & \phantom{a}1.14 & 4.28 $\times 10^{-2}$ \\
LM 1DoF & 49.50 & 28.18 & 0.20 & \phantom{a}0.60 & \phantom{a}1.70 & 2.67 $\times 10^{-2}$ \\
Theia \cite{theia-manual} & \phantom{a}3.46 & \phantom{a}1.28 & 3.28 & 13.92 & 34.62 & 3.72 $\times 10^{-2}$ \\
Theia$^\text{reg}$ \cite{crandall2012sfm} & 11.77 & \phantom{a}1.03 & 7.09 & 20.69 & 40.40 & 3.68 $\times 10^{-2}$ \\
Proposed & \phantom{a}\textbf{2.93} & \phantom{a}\textbf{0.93} & 7.94 & \textbf{23.17} & \textbf{45.51} & \textbf{2.41} $\times 10^{-2}$ \\
\hdashline
COLMAP \cite{schoenberger2016sfm} & 34.18 & \phantom{a}1.42 & \textbf{9.63} & 22.59 & 36.09 & 98.52 \\
\bottomrule
\end{tabular}
}
% \vspace{-3mm}
\label{tbl:lamar_map}
\end{table}
\begin{table}[t]
\centering
\caption{Average and median rot.\ errors ($^\circ$), AUC score at $0.5^\circ$, $1^\circ$, $2^\circ$ and avg.\ run-time (secs) on 1DSfM~\cite{wilson2014robust}.
COLMAP is omitted since the reference comes from SfM.}
% \vspace{-1mm}
\resizebox{0.75\columnwidth}{!}{
\begin{tabular}{l c c c c c c} \toprule
& AVG ($^\circ$) & MED ($^\circ$) & AUC@$0.5^\circ$ &~~~~@$1^\circ~~~~$ &~~~~@$2^\circ~~~~$ & Time (s) \\ \midrule
LM 3DoF & 10.79 & 5.49 & \phantom{a}1.59 & \phantom{a}4.93 & 12.14 & 28.99 \\
LM 1DoF & \phantom{a}3.94 & 1.76 & \phantom{a}4.01 & 11.77 & 26.68 & \phantom{a}\textbf{0.66} \\
LAGO \cite{carlone2014fast} & \phantom{a}9.03 & 2.32 & \phantom{a}2.50 & \phantom{a}7.96 & 19.13 & \phantom{a}5.27 \\
CPL-Sync \cite{fan2019efficient} & \phantom{a}4.82 & 1.88 & \phantom{a}3.45 & 10.24 & 23.41 & 62.17 \\
Theia \cite{theia-manual} & \phantom{a}6.02 & 1.89 & \phantom{a}8.38 & 19.38 & 35.39 & 66.26 \\
Theia$^\text{reg}$ \cite{crandall2012sfm} & \phantom{a}4.82 & 1.38 & \phantom{a}9.03 & 21.28 & 39.17 & 61.65 \\
% Theia$^\text{1DoF}$ & \phantom{a}\textbf{1.53} & \textbf{0.82} & 10.99 & 29.38 & 54.71 & 26.54 \\
Proposed & \phantom{a}\textbf{1.54} & \textbf{0.82} & \textbf{11.02} & \textbf{29.41} & \textbf{54.76} & \phantom{a}4.38 \\
\bottomrule
\end{tabular}
}
% \vspace{-2mm}
\label{tbl:1dsfm}
\end{table}
\begin{table}[t]
\centering
\caption{
Average and median rot.\ errors ($^\circ$), AUC score at $0.5^\circ$, $1^\circ$ and $2^\circ$ on the 1DSfM dataset~\cite{wilson2014robust} with and without the gravity refinement proposed in Section~\ref{sec:refinement}.}
% \vspace{-1mm}
\label{tbl:refinement}
\resizebox{0.7\columnwidth}{!}{
\begin{tabular}{l c c c c c} \toprule
& AVG ($^\circ$) & MED ($^\circ$) & AUC@$0.5^\circ$ &~~~~@$1^\circ~~~~$ &~~~~@$2^\circ~~~~$ \\ \midrule
w/o refinement & 1.60 & 0.88 & \phantom{a}5.51 & 22.61 & 50.89 \\
w/ refinement & \textbf{1.54} & \textbf{0.82} & \textbf{11.02} & \textbf{29.41} & \textbf{54.76}\\
\bottomrule
\end{tabular}
}
% \vspace{-3mm}
\end{table}
% \vspace{-2mm}
\section{Conclusion}
We propose a principled algorithm that leverages the gravity direction to improve camera orientation estimation in rotation averaging for global SfM.
The approach is based on circular statistics and simplifies to an iterative least-squares approach, providing similar convergence guarantees as ordinary linear regression.
By combining it with a robust loss function, we achieve significantly more accurate results than the baselines on four large-scale and real-world datasets.
It also supports scenarios where only a subset of cameras have known gravity.
Additionally, we propose a mechanism for refining gravity.
The proposed method runs several times faster than the widely used Theia library while reducing the error by a large margin.
Compared to LAGO, solving 1-DoF rotation averaging, the proposed algorithm is significantly more robust.
The code will be made public.
This supplementary material provides more results on the LaMAR datasets,
an alternative solution to the problem based on changing the standard 3-DoF optimization,
analysis of the period estimation,
comparisons with deep learning-based alternatives,
results with gravity-aligned essential matrix,
implementation details for Theia baselines,
and detailed experiment results.
% Additionally, it illustrates how the original 3-DoF optimization can be converted to the proposed problem.
% on the transitional relationship between the proposed methods and baselines.
% Additionally, it provides experiments to analyze the sensitivity of the algorithm.
% The analysis of the iterative process of the period estimation is also represented.
\section{Additional Results on LaMAR}
\input{tex/fig/result_lamar_hololens}
In the main paper, we consider gravity directions only in the data captured by smartphones to test the mixed scenario.
Here, we provide additional results for when HoloLens directions are used instead of the ones captured by smartphones.
Moreover, we show results when both data sources have known gravity directions.
% Images captured by LaMAR~\cite{sarlin2022lamar} are also with gravity information from onboard inertial units.
% Thus, we conducted two additional experiments on the dataset: using only gravity from HoloLens data and using gravity from all images.
% Results of the first experiment can be found in Fig.~\ref{fig:result_lamar_hololens}.
The results considering gravity only in the HoloLens sequences are in Fig.~\ref{fig:result_lamar_hololens}.
Planar pose graph optimization methods \cite{carlone2014fast, fan2019efficient} are excluded since they are not applicable in the mixed scenario.
A similar trend can be observed as in the case when only gravity captured by smartphones is known (Table 4 in the main paper).
The proposed method achieves higher AUC scores than Theia-based ones and is about 3-4 times faster.
While the proposed method has a higher average error than the algorithm implemented in Theia~\cite{theia-manual}, it is important to note that the average error is usually not representative in such problems since it is not a robust measure.
In \textit{all} other metrics, the proposed method significantly outperforms Theia.
%
%The proposed method has a higher average error compared to Theia~\cite{theia-manual}.
%However, since it can stem from the small set of images with large errors, having a lower average error does not directly imply that Theia obtains a better estimation.
% This can come from
% however, in practice, this problem is
The proposed method is on par with COLMAP~\cite{schoenberger2016sfm}, having slightly lower AUC scores at $0.5^\circ, 1^\circ$ and a higher AUC score at $2^\circ$.
The proposed method also achieves lower mean and median errors and is faster by several orders of magnitude.
The cumulative distribution functions (CDF) of the absolute rotation errors are also shown in Fig. \ref{fig:result_lamar_hololens}.
The proposed method approaches the top left corner most, indicating its accuracy.
\input{tex/tbl/lamar_map_full}
The results for the case when gravity is available for both types of images are in Table~\ref{tbl:lamar_map_full}.
% For the second experiment, results are summarized in Table~\ref{tbl:lamar_map_full}.
We exclude approximately $150$ images in total from this experiment for which the dataset does not provide gravity directions, and we keep only the largest connected component.
%In this experiment, the gravity of about 50 images on average is missing from the dataset.
This is necessary to test \cite{carlone2014fast, fan2019efficient}.
%These images are excluded, otherwise, \cite{carlone2014fast, fan2019efficient} cannot be applied.
% In this experiment, the proposed method outperforms LAGO~\cite{carlone2014fast} and CPL-Sync~\cite{fan2019efficient} by a large margin.
Despite our tuning efforts, LAGO~\cite{carlone2014fast} and CPL-Sync~\cite{fan2019efficient} do not achieve reasonable accuracy on this dataset.
The proposed method achieves higher AUC scores at $0.5^\circ$, $1^\circ$, and $2^\circ$ compared with Theia-based methods and is about an order of magnitude faster.
The cumulative distribution functions (CDF) of the absolute rotation errors can be found in Figure~\ref{fig:result_lamar_sm}.
The curve of the proposed method (red) is closest to the top left, implying that it obtains the most accurate estimation.
% From the table,
% Results are summarized in Figure~\ref{fig:result_lamar_sm}.
% From the plot, it can be observed that the results
% For the results
% For the results with
% \todo{NOT FINISHED!!!}
\input{tex/fig/result_full_sm}
\section{Alternative Representation}
\label{sec:alternative_representation}
As mentioned in the main text, an alternative way to achieve 1DoF optimization is to constrain the update step in the numerical optimization so that it only changes the rotation around the $y$-axis.
More precisely, in each optimization step, the updates on rotations are projected to be around the $y$-axis.
Though minimizing the same loss function as the proposed method, the alternative formulation requires solving a larger linear system.
As a result, it achieves similar accuracy at the cost of a significantly longer runtime compared to the proposed method.
% However, it has a significantly longer runtime.
Furthermore, it is not applicable when gravity is partially known.
A detailed analysis of the relationship between the proposed method and such alternative formulation can be found in Section \ref{sec:spectrum_constraints}.
We will call this alternative formulation Proposed$^\text{alt}$.
% We conducted the experiments on the EuRoC~\cite{burri2016euroc}, KITTI~\cite{geiger2013vision} and 1DSfM~\cite{wilson2014robust}.
\input{tex/tbl/runtime}
The cumulative distribution functions (CDF) of the absolute rotation errors are shown in Fig.~\ref{fig:result_full_sm}.
As expected, the curves of the alternative formulation and the proposed methods nearly overlap, indicating that they obtain similar results.
The runtimes are summarized in Table~\ref{tbl:runtime}.
The proposed formulation, directly optimizing on the manifold, leads to a consistent 4-9 times speedup compared to the alternative method.
% By formulating the optimization directly on the manifold as proposed, the runtime is much smaller, featuring 4-9 times speedups.
% Results are summarized in the Table~\ref{tbl:runtime}.
% Only the runtimes are reported as the accuracy of the proposed method and such formulation are similar.
% As one can observe, by formulating the optimization directly on the manifold, the runtime is much shorter, featuring 4-9 times speedup.
\section{Spectrum of Constraints}
\label{sec:spectrum_constraints}
% \todo{Tell people what you will discuss here.}
In this section, we further inspect the formulation that incorporates gravity as an extra penalty term.
We show how it allows a spectrum of problems, varying from unconstrained to hard-constrained, and how it can be converted into the proposed method under a specific condition.
Recall the objective function with an extra penalty term (Eq. 21 in the main paper) is formulated as
\begin{equation}
\begin{split}
\arg\min_{\{\bm{R}_i\}} \sum_{(i,j)\in \mathcal{E}} \rho\left(d(\bm{R}_j^\top \tilde{\bm{R}}_{ij} \bm{R}_i, \bm{I})^p\right)
+ \sum_{i} \lambda \rho\left(d(\theta \cdot \left(\bm{v}_i - (\bm{v}_i^\top\bm{g}_i)\bm{g}_i)\right)^p\right).
\label{eq:R_err_penalty_full}
\end{split}
\end{equation}
Since $d_{\text{geod}}(\bm{R},\bm{S}) \approx \|\theta_R\bm{v}_R - \theta_S\bm{v}_S\|_2$, $\|\theta\cdot(\bm{v}_i - (\bm{v}_i^\top\bm{g}_i)\bm{g}_i)\|_2$ approximates the angle distance of $\bm{R}_i$ from the closest upright rotation.
Thus, the proposed formulation is similar to the penalty in \cite{crandall2012sfm}.
Parameter $\lambda$ in Eq.~\eqref{eq:R_err_penalty_full} controls the weighting between consistency with the relative pose and the measured gravity.
When $\lambda$ is 0, the second part of the objective function can be factored out, and Eq.~\eqref{eq:R_err_penalty_full} restores the original 3DoF optimization.
Naturally, as $\lambda$ increases, more emphasis is put on the consistency with measured gravity.
As a result, varying the level of $\lambda$ forms a spectrum of problems, ranging from soft-constrained to hard-constrained ones.
% $0 < \lambda < \infty$ correspondends to the case
Worth noting is the case when $\lambda=\infty$.
The optimal solution in this case should possess the following property:
\begin{equation}
\left\|\theta \cdot\left(\bm{v}_i - (\bm{v}_i^\top\bm{g}_i)\bm{g}_i\right)\right\| = 0.
\end{equation}
To fulfill the above condition, the downward direction in the estimated rotation should be the same as the measured gravity.
% needs to have gravity direction strictly following the prior.
% Thus, if an iterative method is applied to solve the optimization problem, it falls back to the case that the update should only be around gravity direction, which is one of the baselines we compared within the main text.
% Thus, if an iterative method is applied to solve the optimization problem, it proceeds by projecting the update to be upright, and
If an iterative process solves the optimization, in each step, the updates are projected to act around the gravity direction.
As a result, the problem becomes the same as what is presented in Section~\ref{sec:alternative_representation}.
\input{tex/fig/sensitivity}
Besides varying the problem from non-constrained to hard-constrained ones, the above formulation also bridges the proposed method to the original 3DoF optimization.
% Beyond the transitional relationship between the compared baselines, the above formulation can also be used to bridge the proposed method.
When $\lambda = \infty$, suppose the rotation are pre-aligned that gravity direction $\bm{g}_i$ is $[0, 1, 0]^\top$.
Plugging this into the optimization process gives that $\bm{v}^t_x = \bm{v}^t_z = 0$.
This means the $x,z$ components remain 0 across the optimization process, thus the linear systems corresponding to these two rotation components become redundant.
Removing them results in a smaller linear system equivalent to the proposed system in the paper.
% The circular property is implicitly encoded into the optimization process that the angle-axis is converted to rotation matrices when computing residuals.
This observation also directly suggests the efficiency of our methods.
While our method reduces the number of equations in the linear system by $\sim 2m$ where $m$ is the number of pairs, formulation with soft penalty adds $\sim 2n$ additional equations where $n$ is the number of images.
% While our method is an equivalent form of the reduced linear system, formulation with soft penalty adds $~2n$ additional equations.
% To summarize, transitional relationships between different problem formulations can be established as follows: $\lambda = 0$ is the original 3 DoF objective; $0 < \lambda < \infty$ is to add a soft penalty to the objective; $\lambda = \infty$ corresponds to the case where the rotation axis is fixed to be upright.
% Notice that when $\lambda = \infty$ and relative poses are gravity aligned, $x, z$ components in axis-angle representations of the rotations all become 0.
% The corresponding linear systems constantly hold ($\bm{A}\bm{x} = 0$) and become redundant.
% Removing them results in a smaller linear system equivalent to the proposed system in the paper.
% The circular property is implicitly encoded into the optimization process that the angle-axis is converted to rotation matrices when computing residuals.
% Such transitional relationships directly suggest the efficiency of our methods.
% While our method is an equivalent form of the reduced linear system, formulation with soft penalty adds $~2n$ additional equations.
% We show better performance in both runtime and accuracy of our method on synthetic data in Fig. \ref{fig:runtime_baseline}.
\section{Sensitivity Analysis}
% \subsubsection{Sensitivity Analysis}
% To analyze the sensitivity of the method concerning the noise, we report the AUC score as a function of gravity noise and outlier ratio.
In this section, we analyze the sensitivity of the method in the presence of noise and outliers.
In particular, we report the AUC score as a function of gravity noise and the outlier ratio in the pre-estimated relative poses.
%
% The result can be found in Fig.~\ref{fig:sensitivity}.
% In summary, the proposed method works better when the gravity is accurate and the relative pose is error-prone.
% For the left figure, the noise level in gravity is alternated on KITTI~\cite{geiger2013vision}.
% To analyze the sensitivity of the method to gravity noise, we conduct experiments by alternating the input gravity on EuRoC~\cite{burri2016euroc}, KITTI~\cite{geiger2013vision}, and LaMAR~\cite{sarlin2022lamar}.
% More concretely, gravity directions are interpolated between measured and ground truth gravity with different weights.
To analyze the sensitivity of the method to gravity noise, we alternate the level of noise in gravity by interpolating between measured and ground-truth gravity.
More concretely, it is formulated as
%
\begin{equation}
\bm{g}_i^{\alpha} = \frac{\alpha\cdot\bm{g}_i + (1-\alpha)\cdot\bm{g}_i^{gt}}{\left\|\alpha\cdot\bm{g}_i + (1-\alpha)\cdot\bm{g}_i^{gt}\right\|},
\end{equation}
%
where $\bm{g}_i$ is the measured gravity, $\bm{g}_i^{gt}$ is the ground truth gravity, and $\alpha$ controls the level of noise.
Such a setting allows for progressively mixing the noise in the measurements to the ground truth direction.
We vary $\alpha$ from 0.5 to 5, and results are summarized in Figure~\ref{fig:sensitivity_grav}.
The $x$-axis in the plot is the median gravity error, and $\alpha = 1$ corresponds to $\sigma_\text{grav} = 0.54^\circ$, $0.20^\circ$ and $0.31^\circ$ respectively.
% From the figure, we can see that having rough gravity, either in the form of additional error terms or reducing the complexity of the system, can result in more accurate estimations than the original 3DoF optimization.
From the figure, it can be observed that methods incorporating gravity, \ie the proposed method and Theia with extra penalty term (Theia$^\text{reg}$), produce more accurate results compared to the original 3DoF optimization.
This finding further motivates the use of gravity in the rotation averaging problem.
Additionally, the proposed method obtains the most accurate estimations among all methods.
% Additionally, the proposed method generally outperforms Theia$^\text{reg}$.
This holds until $\sim 0.7^\circ$ on the datasets we tested.
Such levels of accuracy can be satisfied by commercial sensors as the median gravity errors on EuRoC~\cite{burri2016euroc} and KITTI~\cite{geiger2013vision} are $0.54^\circ$ and $0.20^\circ$, respectively.
On LaMAR~\cite{sarlin2022lamar}, the median error of HoloLens images is $0.34^\circ$, and that of smartphone images is 0.26$^\circ$.
% for the tested datasets are $0.54^\circ$ (EuRoC~\cite{burri2016euroc}), $0.22^\circ$ (KITTI~\cite{geiger2013vision}), $0.34^\circ$ (LaMAR~\cite{sarlin2022lamar}, HoloLens), and 0.27 (LaMAR~\cite{sarlin2022lamar}, smartphone) respectively.
% \todo{check noise in LAMAR seperately}
% For the figure on the right, we apply the grid setting as in the synthetic experiments.
To analyze the sensitivity of the method to outliers, we conducted synthetic experiments with varying levels of outliers.
There, we apply the same grid setting as in the synthetic experiments in the main paper.
For a brief recap, we assume that the cameras form a 2D grid, and each camera is connected with the closest 24 grid neighbors.
In the experiments, the noise level for gravity prior is fixed to be $0.25^\circ$ which is comparable to that of gravity measurements.
As for the relative pose, we add zero-mean noise by applying random rotations with a standard deviation of $1^\circ$.
To inject outliers, a uniformly randomly drawn set of relative poses is replaced by random rotations.
Results are summarized in Figure~\ref{fig:sensitivity_outlier}.
The plot shows that the performance of the proposed method and baselines are similar when the relative poses are outlier-free.
% Such consistency largely changes as the outlier ratio increases.
% As the outlier ratio increases, different methods illustrate highly different patterns.
As the outlier ratio increases, the accuracy in estimation for all methods degrades at different rates.
% the change in performances across method
% the performances largely change.
LAGO~\cite{carlone2014fast} is subject to drastic performance drop as the outlier ratios increase, validating the claim that its heuristic approach for fixing the period is not robust.
CPL-Sync~\cite{fan2019efficient} is similarly sensitive to outliers.
% On the contrary, Theia-based baselines and our methods all manifest a certain level of robustness in the presence of outliers but the proposed method is the most robust algorithm concerning relative pose error.
In contrast, Theia-based baselines and our method are robust in the presence of outliers.
Among all, the proposed method undergoes the least performance drop as the outlier ratio increases.
For the proposed method, the AUC score at $1^\circ$ only decreases by $11\%$ while those for Theia and Theia with penalty terms drop by $32\%$ and $24\%$, respectively.
\section{Effect of Iteration on Period Estimation}
LAGO~\cite{carlone2014fast} estimates periods for edges from a coarse initialization.
Such a heuristic approach often fails on different datasets.
For example, on the LaMAR dataset, the AUC score of LAGO at $1^\circ$ is only 0.94, while that of the proposed is 23.85, and on 1DSfM, the scores are 7.96 and 29.41, respectively.
We conduct experiments on 1DSfM~\cite{wilson2014robust}, EuRoC~\cite{burri2016euroc}, KITTI~\cite{geiger2013vision}, and LaMAR~\cite{sarlin2022lamar} to analyze the accuracy of period estimation (\textit{correct ratio}) for LAGO~\cite{carlone2014fast} and the proposed iterative solution.
Also, we analyze the relationship between period estimation and accuracy.
% To obtain the ground truth
To calculate the ground truth periods, first, the ground truth rotations are globally aligned with the estimated results and are adjusted to be at most $\pi$ radian to the estimation.
Then, ground truth periods for edges are calculated using the same formulation as in LAGO~\cite{carlone2014fast} with the adjusted ground truth rotations.
% The accuracy for LAGO is calculated as the ratio of periods that are consistent with the initial estimation.
% As for the proposed method, the comparison is ma
% As for the proposed method, period w
% As for the
Results are in Fig.~\ref{fig:ks_ite}.
For iteration 0, the rotations for our method are initialized to be identity.
% As LAGO is not iterative by nature, we use the initial estimation as the result across the iterations.
As LAGO is not an iterative method, we report its error as a constant number across iterations.
From the plot, one can observe that the \textit{correct ratio} is monotonically increasing for the proposed method.
Also, the convergence rate is generally fast and the proposed method can achieve almost 100\% \textit{correct ratio} within 5 iterations.
This indicates the effectiveness of the proposed mechanism.
While for LAGO~\cite{carlone2014fast}, though we can observe it obtain high \textit{correct ratio} on the EuRoC~\cite{burri2016euroc} and KITTI~\cite{geiger2013vision} datasets, which are within the domain the method was originally tailored for, its performance largely drops on other datasets.
% After deviating from sequential image streams for which the method was initially tailored, the estimation of the period becomes inaccurate.
As a result, LAGO fails.
% After deviating from the domain, the method was originally tailored for, the estimation of the period becomes inaccurate, which poses a significant challenge to the robustness of the algorithm.
% Moreover, we find it interesting that estimating the period from coarse initialization is not necessarily better than estimating from random initialization, as the initial correct ratio of LAGO is not better than the proposed method at iteration 0.
The right plot of Fig. \ref{fig:ks_ite} shows the relationship between estimation accuracy and iteration.
By reading the \textit{correct ratio} from the left plot and the AUC score on the right plot, one can notice a strong positive correlation between these two variables.
% only shifting the iteration for 1.
% This is the expected result as iteration estimation relies on the period estimation of the previous iteration.
Across iterations, the accuracy of the rotation orientation increases, leading to a more accurate estimation of periods.
These estimations can later contribute to a more accurate rotation estimation, and finally, the estimation converges to the desired result.
% Surprisingly, one can observe that the
% To verify the performance of the proposed algorithm
% In this section, we show such a heuristic approach may fail on different datasets.
% --------------------------------------------
\begin{figure}[t]
\centering
\subfloat[accuracy of period and camera pose estimation across iterations. $k$ is the period number] {
\begin{tabular}{cc}
\multicolumn{2}{c}{\includegraphics[width=0.65\columnwidth]{figure/legend_k.png}}\\
\includegraphics[height=0.25\columnwidth]{figure/ks_ite_acc.png} &
\includegraphics[height=0.25\columnwidth]{figure/ks_ite_auc.png} \\
\end{tabular}
\label{fig:ks_ite}
}
\subfloat[effect of fixing period]{
\includegraphics[height=0.25\columnwidth]{figure/result_period.png}
\label{fig:fix_period}
}
\caption{
Period estimation can have a large impact on the accuracy of rotation averaging.
}
\end{figure}
Fig.~\ref{fig:fix_period} shows the importance of alternating period estimation in the optimization as proposed in the main paper.
As from Fig.~\ref{fig:ks_ite}, the estimations of periods on EuRoC~\cite{burri2016euroc} and KITTI~\cite{geiger2013vision} are error-free, we only show the result on the LaMAR~\cite{sarlin2022lamar} dataset.
From the plot, it can be seen that the accuracy of the rotation averaging is largely affected by the accuracy of initial period estimation, even if a robust estimation scheme is deployed.
This indicates the necessity of deploying the circular regression.
% --------------------------------------------
\section{Comparison with Learning-based Methods}
\subsection{Unordered Image Collections}
\input{tex/tbl/ra_1dsfm}
In this section, we compare with the recent learning-based DMF-SYNCH~\cite{tejus2023rotation}, NeuRoRA~\cite{purkait2020neurora}, MSP~\cite{yang2021end} and PoGO-Net~\cite{li2021pogo}.
Results on the 1DSfM dataset are reported in Table \ref{tbl:ra_1dsfm}.
% Median errors for perspective image collections are reported in Table~\ref{tbl:ra_1dsfm}.
The results in the table are directly taken from the respective papers.
Given that we do not have a measured gravity direction, similarly as in the main paper, we take the gravity from the ground truth rotations and add zero-mean Gaussian noise with 0.5$^\circ$ standard deviation.
The most accurate learning-based method is PoGO-Net.
Over the scenes where results are provided (all, except for Gendarmenmarkt), the average rotation error of PoGO-Net is 0.96$^\circ$, while that of the proposed method is 0.79$^\circ$.
On Gendarmenmarkt, PoGO-Net results are missing due to its high memory requirement, while the proposed method leads to significantly more accurate results than others.
Given that the gravity error is lower than 0.5$^\circ$ on most of the tested datasets, we expect even larger differences in practice.
% For challenging collections such as Gendarmenmarkt (GDM), the estimation with gravity is significantly closer to ground truth.
%This implies the potential of the algorithm to achieve highly accurate results under challenging conditions.
% Regarding the
\subsection{Sequential Image Collections}
On the KITTI~\cite{geiger2013vision} and EuRoC~\cite{burri2016euroc} datasets, we failed to achieve reasonable results for DMF-SYNCH~\cite{tejus2023rotation}, NeuRoRA~\cite{purkait2020neurora} and MSP~\cite{yang2021end} even when using the code provided by the authors and retraining their models.
As DMF-SYNCH is a method based on matrix completion, it requires not too sparsely connected graphs, which sequential datasets generally do not satisfy.
For the shortest sequence 04 in KITTI, which only contains 277 images, DMF-SYNCH can achieve a median error at the level of 10$^\circ$.
But for sequences with more than 1000 images, we failed to achieve a median error below 30$^\circ$.
Also, the parameters to learn grow quadratically with the number of images, making the method impractical for sequences with more than a few thousand images.
The sparsity also poses challenges to NeuRoRA~\cite{purkait2020neurora} and MSP~\cite{yang2021end}, which are graph-neural-network-based methods.
They contain two parts: a view-graph cleaning component and a rotation refinement component.
% Though based on different inputs, both works design the first component to reject outliers and to refine relative poses.
The first component in both works rejects outliers and refines relative rotations.
Then, NeuRoRA and MSP initialize global rotation estimation via a minimal spanning tree on the input view graph.
% For NeuRoRA, the first part is achieved by a network called CleanNet, which rejects outliers and refines inliers; for MSP, it relies on the
% to predict whether each edge is an outlier and refine the relative pose.
After coarse initialization, they both proceed with a network called FineNet, which refines the rotations.
We attempted to run NeuRoRA~\cite{purkait2020neurora} in different ways.
We tried the weights provided by the authors, finetuning the network on 1DSfM, finetuning on KITTI, and finetuning on EuRoC.
All such tests led to inaccurate results.
%
To eliminate the effect of the potentially failing first step (\ie, view-graph cleaning), we tried providing the ground truth outlier information as initialization for the refinement module.
Since the remaining relative poses are generally accurate, the obtained initialization is reasonable, with median errors generally below 3$^\circ$ for EuRoC and KITTI sequences.
Despite this accurate initialization, FineNet consistently (with all training strategies) reduced the accuracy instead of improving it.
%
%Afterward, we feed the initialization to the FineNet trained using different strategies.
% We assume the view-graph cleaning part works perfectly by using groud-truth outlier information in the initialization.
% We follow the same initialization step
% Then we follow the initialization step as in both works, and it gives reasonable results, with a median error around below 3$^\circ$.
%However, instead of improving the estimation, FineNet trained in the above ways only reduces the accuracy of the initialization.
% the output of the FineNet gives a worse estimation.
We conclude that these two methods are not suitable for the rotation averaging on KITTI, EuRoC, and LaMAR datasets featuring long sequential image streams.
\input{tex/tbl/KITTI_pogonet}
%
As PoGO-Net~\cite{li2021pogo} has no official code available, we report the errors on the KITTI dataset in Table~\ref{tbl:KITTI_pogonet} as provided in the original paper.
The proposed method substantially improves upon PoGO-Net on both sequences.
\section{Results with Gravity-Aligned Essential Matrix}
The results when using 3-DoF and 5-DoF relative poses as input are reported in Table~\ref{tbl:3dof_E}.
The 3-DoF essential matrix solver is incorporated within the same LO-RANSAC framework used in other experiments to estimate gravity-aligned essential matrix.
We allow for full 5 DoF optimization in the essential matrix estimation.
We can see that the gravity-aligned essential matrix improves almost all results, except for Theia’s AVG error on KITTI.
The proposed method is still significantly more accurate than Theia~\cite{theia-manual}.
\input{tex/tbl/3dof_E}
\section{Implementation Details for Theia Baselines}
For the ease of implementation of Theia with regularization terms, we pre-align the camera rotations with the gravity prior.
Thus, the estimated rotations are around $(0, 1, 0)^\top$.
In this case, the first and the last term for camera rotations should be 0.
We can directly penalize these two terms as the deviation from the prior.
After appending these linear systems to the system, the problem can be solved as in Theia~\cite{theia-manual}.
\section{Detailed Experiment Results}
Per sequence result for EuRoC~\cite{burri2016euroc} and KITTI~\cite{geiger2013vision} for the proposed method are summarized in Table~\ref{tbl:full_euroc}, \ref{tbl:full_kitti}.
From the tables, one can observed that the proposed method achieves accurate rotation estimation consistently across the datasets.
\input{tex/tbl/sm_full}
% \todo{Add the per sequence results here}
% We show the results reported in the original paper for PoGO-Net~\cite{li2021pogo}, as no official code is available.
% We directly compare the proposed method with the reported results in the paper.
%Results on KITTI sequences are summarized.
% From the table, one can observe that the proposed method outperforms PoGO-Net~\cite{li2021pogo} by a large margin.
\begin{figure}[h]
\centering
\includegraphics[width=0.4\columnwidth]{figure/result_lamar_map_abs.png}
\resizebox{0.6\columnwidth}{!}{
\begin{tabular}{l c c c c c c} \toprule
& AVG ($^\circ$) & MED ($^\circ$) & AUC@$0.5^\circ$ & @$1^\circ$ & @$2^\circ$ & Time (h) \\ \midrule
LM 3DoF & 52.30 & 32.36 & 0.07 & \phantom{a}0.35 & \phantom{a}1.26 & 4.08 $\times 10^{-2}$ \\
LM 1DoF & 44.90 & 31.61 & 0.28 & \phantom{a}0.73 & \phantom{a}1.81 & 1.69 $\times 10^{-2}$ \\
Theia \cite{theia-manual} & \phantom{a}\textbf{3.46} & \phantom{a}1.28 & 3.28 & 13.92 & 34.62 & 3.88 $\times 10^{-2}$ \\
Theia$^\text{reg}$ \cite{crandall2012sfm} & \phantom{a}8.14 & \phantom{a}1.13 & 5.36 & 17.65 & 39.42 & 4.31 $\times 10^{-2}$ \\
Proposed & \phantom{a}7.19 & \phantom{a}\textbf{1.06} & 7.63 & 20.46 & \textbf{42.02} & \textbf{1.21} $\times 10^{-2}$ \\
\hdashline
COLMAP \cite{schoenberger2016sfm} & 34.18 & \phantom{a}1.42 & \textbf{9.63} & \textbf{22.59} & 36.09 & 98.52 \\
\bottomrule
\end{tabular}
}
\caption{Results for on LaMAR~\cite{sarlin2022lamar}. Only images taken by Hololens data are with gravity, thus \cite{carlone2014fast, fan2019efficient} are not applicable.}
\label{fig:result_lamar_hololens}
\end{figure}
\begin{table}[h]
\centering
\caption{Average and median rotation errors ($^\circ$), the AUC score at $0.5^\circ$, $1^\circ$, and $2^\circ$ and average run-time (hour) on LaMAR~\cite{sarlin2022lamar}. Images without gravity are excluded.
}
\resizebox{0.6\columnwidth}{!}{
\begin{tabular}{l c c c c c c} \toprule
& AVG ($^\circ$) & MED ($^\circ$) & AUC@$0.5^\circ$ & @$1^\circ$ & @$2^\circ$ & Time (h) \\ \midrule
LM 3DoF & 52.31 & 32.20 & \phantom{a}0.08 & \phantom{a}0.36 & \phantom{a}1.25 & 4.05 $\times 10^{-2}$ \\
LM 1DoF & 46.25 & 35.76 & \phantom{a}0.26 & \phantom{a}0.78 & \phantom{a}1.98 & \textbf{1.28 $\times 10^{-3}$} \\
LAGO \cite{carlone2014fast} & 51.28 & 37.17 & \phantom{a}0.33 & \phantom{a}0.94 & \phantom{a}2.22 & 5.18 $\times 10^{-3}$ \\
CPL-Sync \cite{fan2019efficient} & 11.57 & \phantom{a}7.95 & \phantom{a}0.92 & \phantom{a}2.63 & \phantom{a}6.27 & 3.09 $\times 10^{-3}$ \\
Theia \cite{theia-manual} & \phantom{a}3.52 & \phantom{a}1.28 & \phantom{a}3.26 & 13.82 & 34.36 & 4.03 $\times 10^{-2}$ \\
Theia$^\text{reg}$ \cite{crandall2012sfm} & \phantom{a}4.26 & \phantom{a}1.04 & \phantom{a}7.12 & 20.91 & 41.99 & 3.79 $\times 10^{-2}$ \\
% Theia$^\text{1DoF}$ & \phantom{a}2.95 & \phantom{a}0.92 & \phantom{a}9.28 & 24.19 & \textbf{46.70} & 3.16 $\times 10^{-2}$ \\
Proposed & \phantom{a}\textbf{2.90} & \phantom{a}0.94 & \phantom{a}9.32 & 23.85 & \textbf{46.24} & 3.88 $\times 10^{-3}$ \\
\hdashline
COLMAP \cite{schoenberger2016sfm} & \phantom{a}9.68 & \phantom{a}\textbf{0.88} & \textbf{12.71} & \textbf{28.38} & 44.22 & 98.52 \\
\bottomrule
\end{tabular}
}
\label{tbl:lamar_map_full}
\end{table}
\begin{figure*}
\centering
\includegraphics[width=0.9\textwidth]{figure/legend_cpl.png}\\
\begin{subfigure}[b]{0.255\textwidth}
\centering
\includegraphics[width=\textwidth]{figure/result_euroc_cpl.png}
\caption{EuRoC \cite{burri2016euroc}}
\label{fig:result_euroc_sm}
\end{subfigure}
\begin{subfigure}[b]{0.23\textwidth}
\centering
\includegraphics[width=\textwidth]{figure/result_kitti_cpl.png}
\caption{KITTI \cite{geiger2013vision}}
\label{fig:result_kitti_sm}
\end{subfigure}
\begin{subfigure}[b]{0.23\textwidth}
\centering
\includegraphics[width=\textwidth]{figure/result_lamar_map_full.png}
\caption{LaMAR \cite{sarlin2022lamar}}
\label{fig:result_lamar_sm}
\end{subfigure}
\begin{subfigure}[b]{0.23\textwidth}
\centering
\includegraphics[width=\textwidth]{figure/result_1dsfm_cpl.png}
\caption{1DSfM \cite{wilson2014robust}}
\label{fig:result_1dsfm_sm}
\end{subfigure}
\vspace{-5px}
\caption{
The cumulative distribution functions (CDFs) of the absolute rotation errors ($^\circ$). Estimated by the Levenberg-Marquardt~\cite{more2006levenberg} method solving the 3-DoF (LM 3DoF) and 1-DoF (LM 1DoF) problems, by LAGO~\cite{carlone2014fast}, by CPL-Sync~\cite{fan2019efficient}, by the rotation averaging in the Theia library~\cite{theia-manual,chatterjee2013efficient}, by Theia with an additional penalty term (Theia$^\text{reg}$ \cite{crandall2012sfm}), by projecting updates to y-axis (Proposed$^\text{alt}$), by COLMAP~\cite{schoenberger2016sfm}, and by the proposed method.
% For EuRoC, KITTI and LaMAR, COLMAP \cite{schoenberger2016sfm} results are also reported.
%
% In the top row,
Curve "Gravity" stands for the approximate upper bound achievable by using gravity direction.
% In the bottom row, curve "Input" visualizes the error of the noisy input relative rotations.
%
A method being accurate is interpreted by its curve close to the top-left corner.
%
}
\label{fig:result_full_sm}
\end{figure*}
\begin{table}[h]
\centering
\caption{
Runtime (s) for the projected method and direct optimization on the manifold. The latter is 4-9 times faster.}
\resizebox{0.6\columnwidth}{!}{
\begin{tabular}{lcccc}
\toprule
& EuRoC~\cite{burri2016euroc} & KITTI~\cite{geiger2013vision} & LaMAR~\cite{sarlin2022lamar} & 1DSfM~\cite{wilson2014robust} \\
% & EuRoC & KITTI & LaMAR & 1DSfM \\
% &\cite{burri2016euroc} & \cite{geiger2013vision} & \cite{sarlin2022lamar} & \cite{wilson2014robust}\\
\midrule
Proposed$^\text{alt}$ & 0.84 & 0.74 & 113.76 & 26.54 \\
Proposed & \textbf{0.20} & \textbf{0.20} & \phantom{a}\textbf{13.98} & \phantom{a}\textbf{4.38} \\
\bottomrule
\end{tabular}
}
\label{tbl:runtime}
\end{table}
\begin{figure*}[t]
\centering
% \begin{subfigure}[b]{0.74\textwidth}
% \centering
\subfloat[median gravity error vs AUC@1$^\circ$. $\alpha=1$ corresponds to $\sigma_{\text{grav}}$ equals 0.54$^\circ$, 0.20$^\circ$, and 0.31$^\circ$ respectively]{%
\includegraphics[width=0.23\textwidth]{figure/sensitivity_euroc_grav.png}
\includegraphics[width=0.23\textwidth]{figure/sensitivity_kitti_grav.png}
\includegraphics[width=0.23\textwidth]{figure/sensitivity_lamar_map_grav.png}
% \caption{median gravity error vs AUC@1$^\circ$. $\alpha=1$ corresponds to $\sigma_{\text{grav}}$ equals 0.54$^\circ$, 0.20$^\circ$, and 0.31$^\circ$ respectively}
% \caption{AUC at $1^\circ$ as a function of gravity noise level}
\label{fig:sensitivity_grav}
}
% \end{subfigure}
% \begin{subfigure}[b]{0.24\textwidth}
\subfloat[outlier vs AUC@1$^\circ$] {
\includegraphics[width=0.23\textwidth]{figure/sensitivity_outlier_cpl.png}
% \caption{Outlier ratio vs AUC@1$^\circ$}
\label{fig:sensitivity_outlier}
% \end{subfigure}
}
\vspace{-5px}
\caption{Sensitivity analysis w.r.t.\ gravity noise and outlier ratio in the measured relative poses.
For the left plots, gravity are synthesized by interpolating between noisy measurements and ground truth with different weights.
For the right plot, synthetic experiments with grid setting are conducted. In these experiments, different level of outlier are tested with fixed noise level on gravity and relative poses.
% \todo{Explain in more details}
}
\label{fig:sensitivity}
\end{figure*}
\begin{table}[t]
\centering
\caption{Results on 1DSfM dataset \cite{wilson2014robust}. Columns marked with $^\dagger$ are from the respective paper.
We use ground truth with 0.5$^\circ$ standard deviation error as gravity.
}
\resizebox{0.6\columnwidth}{!}{
\begin{tabular}{l|cccc|c}
\hline
& DMF-SYNCH$^\dagger$ & NeuRoRa$^\dagger$ & MSP$^\dagger$ & PoGO-Net$^\dagger$ & \multirow{2}{*}{Proposed} \\
& \cite{tejus2023rotation} & \cite{purkait2020neurora} & \cite{yang2021end} & \cite{li2021pogo} \\
\hline
ALM & \phantom{a}1.2 & 1.2 & 1.07 & \textbf{0.85} & 0.90 \\
ELS & \phantom{a}0.8 & 0.6 & 0.83 & \textbf{0.43} & 0.52 \\
GDM & 10.5 & 2.9 & 3.69 & - & \textbf{0.96} \\
MDR & \phantom{a}2.3 & 1.1 & 1.09 & 0.96 & \textbf{0.67} \\
MND & \phantom{a}0.6 & 0.6 & 0.50 & \textbf{0.37} & 0.51 \\
NYC & \phantom{a}1.8 & 1.1 & 1.12 & \textbf{0.88} & 1.03 \\
PDP & \phantom{a}1.0 & 0.7 & 0.76 & 0.81 & \textbf{0.66} \\
PIC & - & 1.9 & 1.80 & 1.75 & \textbf{0.80} \\
ROF & \phantom{a}1.8 & 1.3 & 1.19 & \textbf{0.69} & 0.71 \\
TOL & \phantom{a}2.7 & 1.4 & 1.25 & \textbf{0.43} & 0.64 \\
TFG & - & 2.2 & - & 1.70 & \textbf{0.86} \\
USQ & \phantom{a}4.4 & 2.0 & 1.85 & \textbf{1.25} & 1.52 \\
VNC & \phantom{a}1.6 & 1.5 & 1.10 & 1.44 & \textbf{0.74} \\
YKM & \phantom{a}1.7 & 0.9 & 0.91 & \textbf{0.72} & \textbf{0.72} \\
\hline
\end{tabular}
}
\label{tbl:ra_1dsfm}
\end{table}
\begin{table}[]
\centering
\caption{Mean rotation error ($^\circ$) on sequence 02 and 08 in KITTI~\cite{geiger2013vision}. Results with $^\dagger$ are taken from the original paper.
}
\resizebox{0.37\columnwidth}{!}{
\begin{tabular}{l c c} \toprule
& PoGO-Net~\cite{li2021pogo}$^\dagger$ & Proposed \\ \midrule
KITTI-02 & 1.08 & \textbf{0.46} \\
KITTI-08 & 2.17 & \textbf{0.53} \\
\bottomrule
\end{tabular}
}
\label{tbl:KITTI_pogonet}
\end{table}
\begin{table}[h]
\centering
\caption{Results with relative poses from both 5-DoF solvers and 3-DoF solvers for EuRoC and KITTI dataset.}
\resizebox{0.7\columnwidth}{!}{
\begin{tabular}{c l c c c c c c} \toprule
~~~~~~~~&&& AVG ($^\circ$) & MED ($^\circ$) & AUC@$0.5^\circ$ & ~~~~@$1^\circ$~~~~& ~~~~@$2^\circ$~~~~ \\ \midrule
\multirow{4}{*}{\rotatebox{90}{\textit{EuRoC}}} &
\multirow{2}{*}{Proposed} & 5DoF & 0.65 & 0.35 & 32.68 & 53.89 & 72.11 \\
&& 3DoF & \textbf{0.62} & \textbf{0.34} & \textbf{33.94} & \textbf{56.64} & \textbf{74.12} \\
% \hdashline
\cdashline{2-8}
& \multirow{2}{*}{Theia} & 5DoF & 0.88 & 0.54 & 22.01 & 43.30 & 64.92 \\
&& 3DoF & \textbf{0.77} & \textbf{0.48} & \textbf{23.56} & \textbf{47.05} & \textbf{68.80} \\
\midrule
% \textit{KITTI}\\
\multirow{4}{*}{\rotatebox{90}{\textit{KITTI}}} &
\multirow{2}{*}{Proposed} & 5DoF & \textbf{0.39} & 0.29 & 48.71 & 69.79 & 83.86 \\
&& 3DoF & \textbf{0.39} & \textbf{0.27} & \textbf{50.69} & \textbf{70.51} & \textbf{83.98} \\
% \hdashline
\cdashline{2-8}
&\multirow{2}{*}{Theia} & 5DoF & \textbf{0.68} & 0.64 & 25.96 & 47.48 & 71.77 \\
&& 3DoF & 2.52 & \textbf{0.54} & \textbf{28.79} & \textbf{52.58} & \textbf{72.92} \\
\bottomrule
\end{tabular}
}
% \vspace{-8px}
\label{tbl:3dof_E}
\end{table}
% \vspace{-10px}
\begin{table}[h]
\centering
\caption{
Full results of the proposed method for EuRoC~\cite{burri2016euroc}
}
\resizebox{\columnwidth}{!}{
\begin{tabular}{lccccccccccc}
\toprule
& ~MH\_01~ & ~MH\_02~ & ~MH\_03~ & ~MH\_04~ & ~MH\_05~ & ~V1\_01~ & ~V1\_02~ & ~V1\_03~ & ~V2\_01~ & ~V2\_02~ & ~V2\_03~ \\
\midrule
mean & 0.78 & 0.21 & 0.20 & 0.46 & 0.18 & 1.79 & 0.32 & 0.58 & 0.48 & 0.69 & 0.63 \\
median & 0.26 & 0.21 & 0.15 & 0.35 & 0.16 & 1.48 & 0.32 & 0.56 & 0.48 & 0.58 & 0.57 \\
\bottomrule
\end{tabular}}
% \vspace{-2mm}
\label{tbl:full_euroc}
\end{table}
\begin{table}[h]
\centering
\caption{
Full results of the proposed method for KITTI~\cite{geiger2013vision}
}
\resizebox{0.7\columnwidth}{!}{
\begin{tabular}{lccccccccc}
\toprule
& ~~~01~~~ & ~~~02~~~ & ~~~04~~~ & ~~~05~~~ & ~~~06~~~ & ~~~07~~~ & ~~~08~~~ & ~~~09~~~ & ~~~10~~~ \\
\midrule
mean & 0.35 & 0.46 & 0.07 & 0.39 & 0.20 & 0.38 & 0.53 & 0.23 & 0.22 \\
median & 0.32 & 0.40 & 0.06 & 0.35 & 0.15 & 0.24 & 0.30 & 0.19 & 0.20 \\
\bottomrule
\end{tabular}}
% \vspace{-2mm}
\label{tbl:full_kitti}
\end{table}
\begin{thebibliography}{10}
\providecommand{\url}[1]{\texttt{#1}}
\providecommand{\urlprefix}{URL }
\providecommand{\doi}[1]{https://doi.org/#1}
\bibitem{arkit}
Arkit. \url{https://developer.apple.com/documentation/arkit/}
\bibitem{madgwick_filter}
Madgwick filter. \url{https://github.com/bjohnsonfl/Madgwick_Filter}
\bibitem{hololens}
Microsoft hololens. \url{https://www.microsoft.com/en-us/hololens}
\bibitem{ceres_solver}
Agarwal, S., Mierle, K., Team, T.C.S.: {Ceres Solver} (3 2022), \url{https://github.com/ceres-solver/ceres-solver}
\bibitem{arie2012global}
Arie-Nachimson, M., Kovalsky, S.Z., Kemelmacher-Shlizerman, I., Singer, A., Basri, R.: Global motion estimation from point matches. In: International Conference on 3D Imaging, Modeling, Processing, Visualization \& Transmission (2012)
\bibitem{black1996robust}
Black, M.J., Anandan, P.: The robust estimation of multiple motions: Parametric and piecewise-smooth flow fields. Computer vision and image understanding \textbf{63}(1), 75--104 (1996)
\bibitem{brossard2020ai}
Brossard, M., Barrau, A., Bonnabel, S.: Ai-imu dead-reckoning. IEEE Transactions on Intelligent Vehicles \textbf{5}(4), 585--595 (2020)
\bibitem{burri2016euroc}
Burri, M., Nikolic, J., Gohl, P., Schneider, T., Rehder, J., Omari, S., Achtelik, M.W., Siegwart, R.: The euroc micro aerial vehicle datasets. The International Journal of Robotics Research \textbf{35}(10), 1157--1163 (2016)
\bibitem{campos2021orb}
Campos, C., Elvira, R., Rodr{\'\i}guez, J.J.G., Montiel, J.M., Tard{\'o}s, J.D.: Orb-slam3: An accurate open-source library for visual, visual--inertial, and multimap slam. IEEE Transactions on Robotics \textbf{37}(6), 1874--1890 (2021)
\bibitem{carlone2012linear}
Carlone, L., Aragues, R., Castellanos, J.A., Bona, B.: A linear approximation for graph-based simultaneous localization and mapping. In: Robotics: Science and Systems. vol.~7, pp. 41--48. MIT Press Cambridge, MA, USA (2012)
\bibitem{carlone2014fast}
Carlone, L., Aragues, R., Castellanos, J.A., Bona, B.: A fast and accurate approximation for planar pose graph optimization. The International Journal of Robotics Research \textbf{33}(7), 965--987 (2014)
\bibitem{carlone2018convex}
Carlone, L., Calafiore, G.C.: Convex relaxations for pose graph optimization with outliers. IEEE Robotics and Automation Letters \textbf{3}(2), 1160--1167 (2018)
\bibitem{carlone2016planar}
Carlone, L., Calafiore, G.C., Tommolillo, C., Dellaert, F.: Planar pose graph optimization: Duality, optimal solutions, and verification. IEEE Transactions on Robotics \textbf{32}(3), 545--565 (2016)
\bibitem{carlone2014angular}
Carlone, L., Censi, A.: From angular manifolds to the integer lattice: Guaranteed orientation estimation with application to pose graph optimization. IEEE Transactions on Robotics \textbf{30}(2), 475--492 (2014)
\bibitem{carlone2015initialization}
Carlone, L., Tron, R., Daniilidis, K., Dellaert, F.: Initialization techniques for 3d slam: A survey on rotation estimation and its use in pose graph optimization. In: 2015 IEEE international conference on robotics and automation (ICRA). pp. 4597--4604. IEEE (2015)
\bibitem{chang2020gnss}
Chang, L., Niu, X., Liu, T.: Gnss/imu/odo/lidar-slam integrated navigation system using imu/odo pre-integration. Sensors \textbf{20}(17), ~4702 (2020)
\bibitem{chatterjee2013efficient}
Chatterjee, A., Govindu, V.M.: Efficient and robust large-scale rotation averaging. In: Proceedings of the IEEE International Conference on Computer Vision. pp. 521--528 (2013)
\bibitem{chatterjee2017robust}
Chatterjee, A., Govindu, V.M.: Robust relative rotation averaging. IEEE transactions on pattern analysis and machine intelligence \textbf{40}(4), 958--972 (2017)
\bibitem{chum2003locally}
Chum, O., Matas, J., Kittler, J.: Locally optimized ransac. In: Pattern Recognition: 25th DAGM Symposium, Magdeburg, Germany, September 10-12, 2003. Proceedings 25. pp. 236--243. Springer (2003)
\bibitem{crandall2012sfm}
Crandall, D.J., Owens, A., Snavely, N., Huttenlocher, D.P.: Sfm with mrfs: Discrete-continuous optimization for large-scale structure from motion. IEEE transactions on pattern analysis and machine intelligence \textbf{35}(12), 2841--2853 (2012)
\bibitem{cremers2018one}
Cremers, J., Klugkist, I.: One direction? a tutorial for circular data analysis using r with examples in cognitive psychology. Frontiers in psychology \textbf{9}, ~2040 (2018)
\bibitem{cui2017hsfm}
Cui, H., Gao, X., Shen, S., Hu, Z.: Hsfm: Hybrid structure-from-motion. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 1212--1221 (2017)
\bibitem{cui2015global}
Cui, Z., Tan, P.: Global structure-from-motion by similarity averaging. In: Proceedings of the IEEE International Conference on Computer Vision (ICCV) (December 2015)
\bibitem{gtsam}
Dellaert, F., Contributors, G.: borglab/gtsam (May 2022). \doi{10.5281/zenodo.5794541}, \url{https://github.com/borglab/gtsam)}
\bibitem{dellaert2020shonan}
Dellaert, F., Rosen, D.M., Wu, J., Mahony, R., Carlone, L.: Shonan rotation averaging: global optimality by surfing {$SO (p)^{n}$}. In: European Conference on Computer Vision. pp. 292--308. Springer (2020)
\bibitem{eriksson2018rotation}
Eriksson, A., Olsson, C., Kahl, F., Chin, T.J.: Rotation averaging and strong duality. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. pp. 127--135 (2018)
\bibitem{fan2019efficient}
Fan, T., Wang, H., Rubenstein, M., Murphey, T.: Efficient and guaranteed planar pose graph optimization using the complex number representation. In: 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). pp. 1904--1911. IEEE (2019)
\bibitem{feng2021comprehensive}
Feng, L., Chen, S., Zhang, C., Zhang, Y., He, Y.: A comprehensive review on recent applications of unmanned aerial vehicle remote sensing with various sensors for high-throughput plant phenotyping. Computers and Electronics in Agriculture \textbf{182}, 106033 (2021)
\bibitem{filliat2007visual}
Filliat, D.: A visual bag of words method for interactive qualitative localization and mapping. In: Proceedings 2007 IEEE International Conference on Robotics and Automation. pp. 3921--3926. IEEE (2007)
\bibitem{fischler1981random}
Fischler, M.A., Bolles, R.C.: Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Communications of the ACM \textbf{24}(6), 381--395 (1981)
\bibitem{fisher1992regression}
Fisher, N.I., Lee, A.J.: Regression models for an angular response. Biometrics pp. 665--677 (1992)
\bibitem{fraundorfer2010minimal}
Fraundorfer, F., Tanskanen, P., Pollefeys, M.: A minimal case solution to the calibrated relative pose problem for the case of two known orientation angles. In: Computer Vision--ECCV 2010: 11th European Conference on Computer Vision, Heraklion, Crete, Greece, September 5-11, 2010, Proceedings, Part IV 11. pp. 269--282. Springer (2010)
\bibitem{fredriksson2012simultaneous}
Fredriksson, J., Olsson, C.: Simultaneous multiple rotation averaging using lagrangian duality. In: Asian Conference on Computer Vision. pp. 245--258. Springer (2012)
\bibitem{ganan1985bayesian}
Ganan, S., McClure, D.: Bayesian image analysis: An application to single photon emission tomography. Amer. Statist. Assoc pp. 12--18 (1985)
\bibitem{gao2018ldso}
Gao, X., Wang, R., Demmel, N., Cremers, D.: Ldso: Direct sparse odometry with loop closure. In: 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). pp. 2198--2204. IEEE (2018)
\bibitem{geiger2013vision}
Geiger, A., Lenz, P., Stiller, C., Urtasun, R.: Vision meets robotics: The kitti dataset. The International Journal of Robotics Research \textbf{32}(11), 1231--1237 (2013)
\bibitem{gould1969regression}
Gould, A.L.: A regression technique for angular variates. Biometrics pp. 683--700 (1969)
\bibitem{govindu2001combining}
Govindu, V.M.: Combining two-view constraints for motion estimation. In: Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001. vol.~2, pp. II--II. IEEE (2001)
\bibitem{hartley2011l1}
Hartley, R., Aftab, K., Trumpf, J.: L1 rotation averaging using the weiszfeld algorithm. In: CVPR 2011. pp. 3041--3048. IEEE (2011)
\bibitem{hartley2013rotation}
Hartley, R., Trumpf, J., Dai, Y., Li, H.: Rotation averaging. International journal of computer vision \textbf{103}, 267--305 (2013)
\bibitem{heinly2015reconstructing}
Heinly, J., Schonberger, J.L., Dunn, E., Frahm, J.M.: Reconstructing the world* in six days*(as captured by the yahoo 100 million image dataset). In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 3287--3295 (2015)
\bibitem{huber1992robust}
Huber, P.J.: Robust estimation of a location parameter. Breakthroughs in statistics: Methodology and distribution pp. 492--518 (1992)
\bibitem{levenberg1944method}
Levenberg, K.: A method for the solution of certain non-linear problems in least squares. Quarterly of applied mathematics \textbf{2}(2), 164--168 (1944)
\bibitem{li2021pogo}
Li, X., Ling, H.: Pogo-net: pose graph optimization with graph neural networks. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 5895--5905 (2021)
\bibitem{liu2019edge}
Liu, S., Liu, L., Tang, J., Yu, B., Wang, Y., Shi, W.: Edge computing for autonomous driving: Opportunities and challenges. Proceedings of the IEEE \textbf{107}(8), 1697--1716 (2019)
\bibitem{lopez2019deep}
Lopez, M., Mari, R., Gargallo, P., Kuang, Y., Gonzalez-Jimenez, J., Haro, G.: Deep single image camera calibration with radial distortion. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 11817--11825 (2019)
\bibitem{madgwick2010efficient}
Madgwick, S., et~al.: An efficient orientation filter for inertial and inertial/magnetic sensor arrays. Report x-io and University of Bristol (UK) \textbf{25}, 113--118 (2010)
\bibitem{madgwick2011estimation}
Madgwick, S.O., Harrison, A.J., Vaidyanathan, R.: Estimation of imu and marg orientation using a gradient descent algorithm. In: 2011 IEEE international conference on rehabilitation robotics. pp.~1--7. IEEE (2011)
\bibitem{mardia1975statistics}
Mardia, K.V.: Statistics of directional data. Journal of the Royal Statistical Society: Series B (Methodological) \textbf{37}(3), 349--371 (1975)
\bibitem{marquardt1963algorithm}
Marquardt, D.W.: An algorithm for least-squares estimation of nonlinear parameters. Journal of the society for Industrial and Applied Mathematics \textbf{11}(2), 431--441 (1963)
\bibitem{martinec2007robust}
Martinec, D., Pajdla, T.: Robust rotation and translation estimation in multiview reconstruction. In: 2007 IEEE Conference on Computer Vision and Pattern Recognition. pp.~1--8. IEEE (2007)
\bibitem{more2006levenberg}
Mor{\'e}, J.J.: The levenberg-marquardt algorithm: implementation and theory. In: Numerical Analysis: Proceedings of the Biennial Conference Held at Dundee, June 28--July 1, 1977. pp. 105--116. Springer (2006)
\bibitem{moulon2013global}
Moulon, P., Monasse, P., Marlet, R.: Global fusion of relative motions for robust, accurate and scalable structure from motion. In: Proceedings of the IEEE international conference on computer vision. pp. 3248--3255 (2013)
\bibitem{moulon2016openmvg}
Moulon, P., Monasse, P., Perrot, R., Marlet, R.: Open{MVG}: Open multiple view geometry. In: International Workshop on Reproducible Research in Pattern Recognition (2016)
\bibitem{mur2017orb}
Mur-Artal, R., Tard{\'o}s, J.D.: Orb-slam2: An open-source slam system for monocular, stereo, and rgb-d cameras. IEEE transactions on robotics \textbf{33}(5), 1255--1262 (2017)
\bibitem{olsson2011stable}
Olsson, C., Enqvist, O.: Stable structure from motion for unordered image collections. In: Scandinavian Conference on Image Analysis. pp. 524--535. Springer (2011)
\bibitem{petovello2003real}
Petovello, M.G.: Real-time integration of a tactical-grade IMU and GPS for high-accuracy positioning and navigation. Citeseer (2003)
\bibitem{purkait2020neurora}
Purkait, P., Chin, T.J., Reid, I.: Neurora: Neural robust rotation averaging. In: European Conference on Computer Vision. pp. 137--154. Springer (2020)
\bibitem{qin2019general}
Qin, T., Cao, S., Pan, J., Shen, S.: A general optimization-based framework for global pose estimation with multiple sensors. arXiv preprint arXiv:1901.03642 (2019)
\bibitem{sarlin2022lamar}
Sarlin, P.E., Dusmanu, M., Sch{\"o}nberger, J.L., Speciale, P., Gruber, L., Larsson, V., Miksik, O., Pollefeys, M.: Lamar: Benchmarking localization and mapping for augmented reality. In: Computer Vision--ECCV 2022: 17th European Conference, Tel Aviv, Israel, October 23--27, 2022, Proceedings, Part VII. pp. 686--704. Springer (2022)
\bibitem{saurer2015homography}
Saurer, O., Vasseur, P., Demonceaux, C., Fraundorfer, F.: A homography formulation to the 3pt plus a common direction relative pose problem. In: Computer Vision--ACCV 2014: 12th Asian Conference on Computer Vision, Singapore, Singapore, November 1-5, 2014, Revised Selected Papers, Part II 12. pp. 288--301. Springer (2015)
\bibitem{schoenberger2016sfm}
Sch\"{o}nberger, J.L., Frahm, J.M.: Structure-from-motion revisited. In: Conference on Computer Vision and Pattern Recognition (CVPR) (2016)
\bibitem{sharafutdinov2023comparison}
Sharafutdinov, D., Griguletskii, M., Kopanev, P., Kurenkov, M., Ferrer, G., Burkov, A., Gonnochenko, A., Tsetserukou, D.: Comparison of modern open-source visual slam approaches. Journal of Intelligent \& Robotic Systems \textbf{107}(3), ~43 (2023)
\bibitem{sidhartha2021all}
Sidhartha, C., Govindu, V.M.: It is all in the weights: Robust rotation averaging revisited. In: 2021 International Conference on 3D Vision (3DV). pp. 1134--1143. IEEE (2021)
\bibitem{snavely2006photo}
Snavely, N., Seitz, S.M., Szeliski, R.: Photo tourism: exploring photo collections in 3d. In: ACM siggraph 2006 papers, pp. 835--846 (2006)
\bibitem{snavely2008modeling}
Snavely, N., Seitz, S.M., Szeliski, R.: Modeling the world from internet photo collections. International journal of computer vision \textbf{80}, 189--210 (2008)
\bibitem{sumikura2019openvslam}
Sumikura, S., Shibuya, M., Sakurada, K.: Openvslam: A versatile visual slam framework. In: Proceedings of the 27th ACM International Conference on Multimedia. pp. 2292--2295 (2019)
\bibitem{theia-manual}
Sweeney, C.: Theia multiview geometry library: Tutorial \& reference. \url{http://theia-sfm.org}
\bibitem{tejus2023rotation}
Tejus, G., Zara, G., Rota, P., Fusiello, A., Ricci, E., Arrigoni, F.: Rotation synchronization via deep matrix factorization. arXiv preprint arXiv:2305.05268 (2023)
\bibitem{triggs2000bundle}
Triggs, B., McLauchlan, P.F., Hartley, R.I., Fitzgibbon, A.W.: Bundle adjustment—a modern synthesis. In: Vision Algorithms: Theory and Practice: International Workshop on Vision Algorithms Corfu, Greece, September 21--22, 1999 Proceedings. pp. 298--372. Springer (2000)
\bibitem{ullman1979interpretation}
Ullman, S.: The interpretation of structure from motion. Proceedings of the Royal Society of London. Series B. Biological Sciences \textbf{203}(1153), 405--426 (1979)
\bibitem{usenko2019visual}
Usenko, V., Demmel, N., Schubert, D., St{\"u}ckler, J., Cremers, D.: Visual-inertial mapping with non-linear factor recovery. IEEE Robotics and Automation Letters \textbf{5}(2), 422--429 (2019)
\bibitem{wilson2016rotations}
Wilson, K., Bindel, D., Snavely, N.: When is rotations averaging hard? In: ECCV (2016)
\bibitem{wilson2014robust}
Wilson, K., Snavely, N.: Robust global translations with 1dsfm. In: Computer Vision--ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part III 13. pp. 61--75. Springer (2014)
\bibitem{wolsey1999integer}
Wolsey, L.A., Nemhauser, G.L.: Integer and combinatorial optimization, vol.~55. John Wiley \& Sons (1999)
\bibitem{wu2013towards}
Wu, C.: Towards linear-time incremental structure from motion. In: 2013 International Conference on 3D Vision-3DV 2013. pp. 127--134. IEEE (2013)
\bibitem{wu2023decoupled}
Wu, T., Gao, F.: A decoupled and linear framework for global outlier rejection over planar pose graph. In: 2023 IEEE International Conference on Robotics and Automation (ICRA). pp. 4150--4156. IEEE (2023)
\bibitem{yang2021end}
Yang, L., Li, H., Rahim, J.A., Cui, Z., Tan, P.: End-to-end rotation averaging with multi-source propagation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 11774--11783 (2021)
\bibitem{zhang2023revisiting}
Zhang, G., Larsson, V., Barath, D.: Revisiting rotation averaging: Uncertainties and robust losses. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 17215--17224 (2023)
\bibitem{zhu2018largesfm}
Zhu, S., Zhang, R., Zhou, L., Shen, T., Fang, T., Tan, P., Quan, L.: Very large-scale global sfm by distributed motion averaging. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 4568--4577 (2018)
\end{thebibliography}