DeepGCNs: Making GCNs Go as Deep as CNNs
Extracted source text · arXiv 1910.06849
%% bare_jrnl_compsoc.tex
%% V1.4b
%% 2015/08/26
%% by Michael Shell
%% See:
%% http://www.michaelshell.org/
%% for current contact information.
%%
%% This is a skeleton file demonstrating the use of IEEE+tran.cls
%% (requires IEEEtran.cls version 1.8b or later) with an IEEE
%% Computer Society journal paper.
%%
%% Support sites:
%% http://www.michaelshell.org/tex/ieeetran/
%% http://www.ctan.org/pkg/ieeetran
%% and
%% http://www.ieee.org/
%%*************************************************************************
%% Legal Notice:
%% This code is offered as-is without any warranty either expressed or
%% implied; without even the implied warranty of MERCHANTABILITY or
%% FITNESS FOR A PARTICULAR PURPOSE!
%% User assumes all risk.
%% In no event shall the IEEE or any contributor to this code be liable for
%% any damages or losses, including, but not limited to, incidental,
%% consequential, or any other damages, resulting from the use or misuse
%% of any information contained here.
%%
%% All comments are the opinions of their respective authors and are not
%% necessarily endorsed by the IEEE.
%%
%% This work is distributed under the LaTeX Project Public License (LPPL)
%% ( http://www.latex-project.org/ ) version 1.3, and may be freely used,
%% distributed and modified. A copy of the LPPL, version 1.3, is included
%% in the base LaTeX documentation of all distributions of LaTeX released
%% 2003/12/01 or later.
%% Retain all contribution notices and credits.
%% ** Modified files should be clearly indicated as such, including **
%% ** renaming them and changing author support contact information. **
%%*************************************************************************
% *** Authors should verify (and, if needed, correct) their LaTeX system ***
% *** with the testflow diagnostic prior to trusting their LaTeX platform ***
% *** with production work. The IEEE's font choices and paper sizes can ***
% *** trigger bugs that do not appear when using other class files. *** ***
% The testflow support page is at:
% http://www.michaelshell.org/tex/testflow/
\documentclass[10pt,journal,compsoc]{IEEEtran}
%
% If IEEEtran.cls has not been installed into the LaTeX system files,
% manually specify the path to it like:
% \documentclass[10pt,journal,compsoc]{../sty/IEEEtran}
% Some very useful LaTeX packages include:
% (uncomment the ones you want to load)
% *** MISC UTILITY PACKAGES ***
%
%\usepackage{ifpdf}
% Heiko Oberdiek's ifpdf.sty is very useful if you need conditional
% compilation based on whether the output is pdf or dvi.
% usage:
% \ifpdf
% % pdf code
% \else
% % dvi code
% \fi
% The latest version of ifpdf.sty can be obtained from:
% http://www.ctan.org/pkg/ifpdf
% Also, note that IEEEtran.cls V1.7 and later provides a builtin
% \ifCLASSINFOpdf conditional that works the same way.
% When switching from latex to pdflatex and vice-versa, the compiler may
% have to be run twice to clear warning/error messages.
% *** CITATION PACKAGES ***
%
\ifCLASSOPTIONcompsoc
% IEEE Computer Society needs nocompress option
% requires cite.sty v4.0 or later (November 2003)
\usepackage[nocompress]{cite}
\else
% normal IEEE
\usepackage{cite}
\fi
% cite.sty was written by Donald Arseneau
% V1.6 and later of IEEEtran pre-defines the format of the cite.sty package
% \cite{} output to follow that of the IEEE. Loading the cite package will
% result in citation numbers being automatically sorted and properly
% "compressed/ranged". e.g., [1], [9], [2], [7], [5], [6] without using
% cite.sty will become [1], [2], [5]--[7], [9] using cite.sty. cite.sty's
% \cite will automatically add leading space, if needed. Use cite.sty's
% noadjust option (cite.sty V3.8 and later) if you want to turn this off
% such as if a citation ever needs to be enclosed in parenthesis.
% cite.sty is already installed on most LaTeX systems. Be sure and use
% version 5.0 (2009-03-20) and later if using hyperref.sty.
% The latest version can be obtained at:
% http://www.ctan.org/pkg/cite
% The documentation is contained in the cite.sty file itself.
%
% Note that some packages require special options to format as the Computer
% Society requires. In particular, Computer Society papers do not use
% compressed citation ranges as is done in typical IEEE papers
% (e.g., [1]-[4]). Instead, they list every citation separately in order
% (e.g., [1], [2], [3], [4]). To get the latter we need to load the cite
% package with the nocompress option which is supported by cite.sty v4.0
% and later. Note also the use of a CLASSOPTION conditional provided by
% IEEEtran.cls V1.7 and later.
% *** GRAPHICS RELATED PACKAGES ***
%
\ifCLASSINFOpdf
% \usepackage[pdftex]{graphicx}
% declare the path(s) where your graphic files are
% \graphicspath{{../pdf/}{../jpeg/}}
% and their extensions so you won't have to specify these with
% every instance of \includegraphics
% \DeclareGraphicsExtensions{.pdf,.jpeg,.png}
\else
% or other class option (dvipsone, dvipdf, if not using dvips). graphicx
% will default to the driver specified in the system graphics.cfg if no
% driver is specified.
% \usepackage[dvips]{graphicx}
% declare the path(s) where your graphic files are
% \graphicspath{{../eps/}}
% and their extensions so you won't have to specify these with
% every instance of \includegraphics
% \DeclareGraphicsExtensions{.eps}
\fi
% graphicx was written by David Carlisle and Sebastian Rahtz. It is
% required if you want graphics, photos, etc. graphicx.sty is already
% installed on most LaTeX systems. The latest version and documentation
% can be obtained at:
% http://www.ctan.org/pkg/graphicx
% Another good source of documentation is "Using Imported Graphics in
% LaTeX2e" by Keith Reckdahl which can be found at:
% http://www.ctan.org/pkg/epslatex
%
% latex, and pdflatex in dvi mode, support graphics in encapsulated
% postscript (.eps) format. pdflatex in pdf mode supports graphics
% in .pdf, .jpeg, .png and .mps (metapost) formats. Users should ensure
% that all non-photo figures use a vector format (.eps, .pdf, .mps) and
% not a bitmapped formats (.jpeg, .png). The IEEE frowns on bitmapped formats
% which can result in "jaggedy"/blurry rendering of lines and letters as
% well as large increases in file sizes.
%
% You can find documentation about the pdfTeX application at:
% http://www.tug.org/applications/pdftex
% *** MATH PACKAGES ***
%
%\usepackage{amsmath}
% A popular package from the American Mathematical Society that provides
% many useful and powerful commands for dealing with mathematics.
%
% Note that the amsmath package sets \interdisplaylinepenalty to 10000
% thus preventing page breaks from occurring within multiline equations. Use:
%\interdisplaylinepenalty=2500
% after loading amsmath to restore such page breaks as IEEEtran.cls normally
% does. amsmath.sty is already installed on most LaTeX systems. The latest
% version and documentation can be obtained at:
% http://www.ctan.org/pkg/amsmath
% *** SPECIALIZED LIST PACKAGES ***
%
%\usepackage{algorithmic}
% algorithmic.sty was written by Peter Williams and Rogerio Brito.
% This package provides an algorithmic environment fo describing algorithms.
% You can use the algorithmic environment in-text or within a figure
% environment to provide for a floating algorithm. Do NOT use the algorithm
% floating environment provided by algorithm.sty (by the same authors) or
% algorithm2e.sty (by Christophe Fiorio) as the IEEE does not use dedicated
% algorithm float types and packages that provide these will not provide
% correct IEEE style captions. The latest version and documentation of
% algorithmic.sty can be obtained at:
% http://www.ctan.org/pkg/algorithms
% Also of interest may be the (relatively newer and more customizable)
% algorithmicx.sty package by Szasz Janos:
% http://www.ctan.org/pkg/algorithmicx
% *** ALIGNMENT PACKAGES ***
%
%\usepackage{array}
% Frank Mittelbach's and David Carlisle's array.sty patches and improves
% the standard LaTeX2e array and tabular environments to provide better
% appearance and additional user controls. As the default LaTeX2e table
% generation code is lacking to the point of almost being broken with
% respect to the quality of the end results, all users are strongly
% advised to use an enhanced (at the very least that provided by array.sty)
% set of table tools. array.sty is already installed on most systems. The
% latest version and documentation can be obtained at:
% http://www.ctan.org/pkg/array
% IEEEtran contains the IEEEeqnarray family of commands that can be used to
% generate multiline equations as well as matrices, tables, etc., of high
% quality.
% *** SUBFIGURE PACKAGES ***
%\ifCLASSOPTIONcompsoc
% \usepackage[caption=false,font=footnotesize,labelfont=sf,textfont=sf]{subfig}
%\else
% \usepackage[caption=false,font=footnotesize]{subfig}
%\fi
% subfig.sty, written by Steven Douglas Cochran, is the modern replacement
% for subfigure.sty, the latter of which is no longer maintained and is
% incompatible with some LaTeX packages including fixltx2e. However,
% subfig.sty requires and automatically loads Axel Sommerfeldt's caption.sty
% which will override IEEEtran.cls' handling of captions and this will result
% in non-IEEE style figure/table captions. To prevent this problem, be sure
% and invoke subfig.sty's "caption=false" package option (available since
% subfig.sty version 1.3, 2005/06/28) as this is will preserve IEEEtran.cls
% handling of captions.
% Note that the Computer Society format requires a sans serif font rather
% than the serif font used in traditional IEEE formatting and thus the need
% to invoke different subfig.sty package options depending on whether
% compsoc mode has been enabled.
%
% The latest version and documentation of subfig.sty can be obtained at:
% http://www.ctan.org/pkg/subfig
% *** FLOAT PACKAGES ***
%
%\usepackage{fixltx2e}
% fixltx2e, the successor to the earlier fix2col.sty, was written by
% Frank Mittelbach and David Carlisle. This package corrects a few problems
% in the LaTeX2e kernel, the most notable of which is that in current
% LaTeX2e releases, the ordering of single and double column floats is not
% guaranteed to be preserved. Thus, an unpatched LaTeX2e can allow a
% single column figure to be placed prior to an earlier double column
% figure.
% Be aware that LaTeX2e kernels dated 2015 and later have fixltx2e.sty's
% corrections already built into the system in which case a warning will
% be issued if an attempt is made to load fixltx2e.sty as it is no longer
% needed.
% The latest version and documentation can be found at:
% http://www.ctan.org/pkg/fixltx2e
%\usepackage{stfloats}
% stfloats.sty was written by Sigitas Tolusis. This package gives LaTeX2e
% the ability to do double column floats at the bottom of the page as well
% as the top. (e.g., "\begin{figure*}[!b]" is not normally possible in
% LaTeX2e). It also provides a command:
%\fnbelowfloat
% to enable the placement of footnotes below bottom floats (the standard
% LaTeX2e kernel puts them above bottom floats). This is an invasive package
% which rewrites many portions of the LaTeX2e float routines. It may not work
% with other packages that modify the LaTeX2e float routines. The latest
% version and documentation can be obtained at:
% http://www.ctan.org/pkg/stfloats
% Do not use the stfloats baselinefloat ability as the IEEE does not allow
% \baselineskip to stretch. Authors submitting work to the IEEE should note
% that the IEEE rarely uses double column equations and that authors should try
% to avoid such use. Do not be tempted to use the cuted.sty or midfloat.sty
% packages (also by Sigitas Tolusis) as the IEEE does not format its papers in
% such ways.
% Do not attempt to use stfloats with fixltx2e as they are incompatible.
% Instead, use Morten Hogholm'a dblfloatfix which combines the features
% of both fixltx2e and stfloats:
%
% \usepackage{dblfloatfix}
% The latest version can be found at:
% http://www.ctan.org/pkg/dblfloatfix
%\ifCLASSOPTIONcaptionsoff
% \usepackage[nomarkers]{endfloat}
% \let\MYoriglatexcaption\caption
% \renewcommand{\caption}[2][\relax]{\MYoriglatexcaption[#2]{#2}}
%\fi
% endfloat.sty was written by James Darrell McCauley, Jeff Goldberg and
% Axel Sommerfeldt. This package may be useful when used in conjunction with
% IEEEtran.cls' captionsoff option. Some IEEE journals/societies require that
% submissions have lists of figures/tables at the end of the paper and that
% figures/tables without any captions are placed on a page by themselves at
% the end of the document. If needed, the draftcls IEEEtran class option or
% \CLASSINPUTbaselinestretch interface can be used to increase the line
% spacing as well. Be sure and use the nomarkers option of endfloat to
% prevent endfloat from "marking" where the figures would have been placed
% in the text. The two hack lines of code above are a slight modification of
% that suggested by in the endfloat docs (section 8.4.1) to ensure that
% the full captions always appear in the list of figures/tables - even if
% the user used the short optional argument of \caption[]{}.
% IEEE papers do not typically make use of \caption[]'s optional argument,
% so this should not be an issue. A similar trick can be used to disable
% captions of packages such as subfig.sty that lack options to turn off
% the subcaptions:
% For subfig.sty:
% \let\MYorigsubfloat\subfloat
% \renewcommand{\subfloat}[2][\relax]{\MYorigsubfloat[]{#2}}
% However, the above trick will not work if both optional arguments of
% the \subfloat command are used. Furthermore, there needs to be a
% description of each subfigure *somewhere* and endfloat does not add
% subfigure captions to its list of figures. Thus, the best approach is to
% avoid the use of subfigure captions (many IEEE journals avoid them anyway)
% and instead reference/explain all the subfigures within the main caption.
% The latest version of endfloat.sty and its documentation can obtained at:
% http://www.ctan.org/pkg/endfloat
%
% The IEEEtran \ifCLASSOPTIONcaptionsoff conditional can also be used
% later in the document, say, to conditionally put the References on a
% page by themselves.
% *** PDF, URL AND HYPERLINK PACKAGES ***
%
%\usepackage{url}
% url.sty was written by Donald Arseneau. It provides better support for
% handling and breaking URLs. url.sty is already installed on most LaTeX
% systems. The latest version and documentation can be obtained at:
% http://www.ctan.org/pkg/url
% Basically, \url{my_url_here}.
% *** Do not adjust lengths that control margins, column widths, etc. ***
% *** Do not use packages that alter fonts (such as pslatex). ***
% There should be no need to do such things with IEEEtran.cls V1.6 and later.
% (Unless specifically asked to do so by the journal or conference you plan
% to submit to, of course. )
% correct bad hyphenation here
%\hyphenation{op-tical net-works semi-conduc-tor}
\usepackage{times}
\usepackage{epsfig}
\usepackage{graphicx}
\usepackage{amsmath}
\usepackage{amssymb}
\usepackage{color}
% \usepackage{dvipsnames}
% Include other packages here, before hyperref.
\usepackage{hyperref}
% \usepackage[pagebackref=true,breaklinks=true,letterpaper=true,colorlinks,bookmarks=false]{hyperref}
\usepackage{booktabs}
\usepackage{multirow}
\usepackage{xspace}
\newcommand{\etal}{\emph{et al.}\xspace}
\newcommand{\etc}{\emph{etc.}\xspace}
\newcommand{\eg}{\emph{e.g.}\xspace}
\newcommand{\ie}{\emph{i.e.}\xspace}
\newcommand{\todo}[1]{{\color{red}[TODO: #1]}}
\newcommand{\figLabel}{Figure\xspace}
\newcommand{\eqLabel}{Equation\xspace}
\newcommand{\secLabel}{Section\xspace}
\newcommand{\tblLabel}{Table\xspace}
\newcommand{\mysection}[1]{\vspace{3pt}\noindent\textbf{#1.}}
\newcommand{\supp}{{\textbf{supplementary material}}\xspace}
\definecolor{orange}{rgb}{1,0.5,0}
\definecolor{maroon}{rgb}{0.51,0,0}
\newcommand{\res}{$\oplus$}
\newcommand{\dense}{$\rhd$\hspace{-2pt}$\lhd$}
\newcommand{\nc}{}
% for revision difference
\newcommand{\R}[1]{{\color{black}#1}}
%Orient text vertically. Useful for tables and figures.
\def\rot#1{\rotatebox{0}{#1}}
\begin{document}
%
% paper title
% Titles are generally capitalized except for words such as a, an, and, as,
% at, but, by, for, in, nor, of, on, or, the, to and up, which are usually
% not capitalized unless they are the first or last word of the title.
% Linebreaks \\ can be used within to get better formatting as desired.
% Do not put math or special symbols in the title.
\title{DeepGCNs:\\ Making GCNs Go as Deep as CNNs\small\\\url{https://www.deepgcns.org}}
%\\\small\url{ https://sites.google.com/view/deep-gcns}}
%
%
% author names and IEEE memberships
% note positions of commas and nonbreaking spaces ( ~ ) LaTeX will not break
% a structure at a ~ so this keeps an author's name from being broken across
% two lines.
% use \thanks{} to gain access to the first footnote area
% a separate \thanks must be used for each paragraph as LaTeX2e's \thanks
% was not built to handle multiple paragraphs
%
%
%\IEEEcompsocitemizethanks is a special \thanks that produces the bulleted
% lists the Computer Society journals use for "first footnote" author
% affiliations. Use \IEEEcompsocthanksitem which works much like \item
% for each affiliation group. When not in compsoc mode,
% \IEEEcompsocitemizethanks becomes like \thanks and
% \IEEEcompsocthanksitem becomes a line break with idention. This
% facilitates dual compilation, although admittedly the differences in the
% desired content of \author between the different types of papers makes a
% one-size-fits-all approach a daunting prospect. For instance, compsoc
% journal papers have the author affiliations above the "Manuscript
% received ..." text while in non-compsoc journals this is reversed. Sigh.
\author{Guohao Li\thanks{*equal contribution}\textsuperscript{*}\quad Matthias M\"uller\textsuperscript{*} \quad Guocheng Qian \quad Itzel C. Delgadillo \quad Abdulellah Abualshour \quad \\ Ali Thabet\quad Bernard Ghanem\\
Visual Computing Center,~ KAUST,~ Thuwal,~ Saudi Arabia\\
{\tt\footnotesize \{guohao.li, matthias.mueller.2, guocheng.qian, itzel.delgadilloperez, abdulellah.abualshour, ali.thabet, bernard.ghanem\}@kaust.edu.sa}}
% \author{Michael~Shell,~\IEEEmembership{Member,~IEEE,}
% John~Doe,~\IEEEmembership{Fellow,~OSA,}
% and~Jane~Doe,~\IEEEmembership{Life~Fellow,~IEEE}% <-this % stops a space
% \IEEEcompsocitemizethanks{\IEEEcompsocthanksitem M. Shell was with the Department
% of Electrical and Computer Engineering, Georgia Institute of Technology, Atlanta,
% GA, 30332.\protect\\
% % note need leading \protect in front of \\ to get a newline within \thanks as
% % \\ is fragile and will error, could use \hfil\break instead.
% E-mail: see http://www.michaelshell.org/contact.html
% \IEEEcompsocthanksitem J. Doe and J. Doe are with Anonymous University.}% <-this % stops an unwanted space
% \thanks{Manuscript received April 19, 2005; revised August 26, 2015.}}
% note the % following the last \IEEEmembership and also \thanks -
% these prevent an unwanted space from occurring between the last author name
% and the end of the author line. i.e., if you had this:
%
% \author{....lastname \thanks{...} \thanks{...} }
% ^------------^------------^----Do not want these spaces!
%
% a space would be appended to the last name and could cause every name on that
% line to be shifted left slightly. This is one of those "LaTeX things". For
% instance, "\textbf{A} \textbf{B}" will typeset as "A B" not "AB". To get
% "AB" then you have to do: "\textbf{A}\textbf{B}"
% \thanks is no different in this regard, so shield the last } of each \thanks
% that ends a line with a % and do not let a space in before the next \thanks.
% Spaces after \IEEEmembership other than the last one are OK (and needed) as
% you are supposed to have spaces between the names. For what it is worth,
% this is a minor point as most people would not even notice if the said evil
% space somehow managed to creep in.
% The paper headers
% \markboth{Journal of \LaTeX\ Class Files,~Vol.~14, No.~8, August~2015}%
\markboth{IEEE TPAMI, Special Issue on Graphs in Vision and Pattern Analysis. Under review}
{Shell \MakeLowercase{\textit{et al.}}: Bare Demo of IEEEtran.cls for Computer Society Journals}
% The only time the second header will appear is for the odd numbered pages
% after the title page when using the twoside option.
%
% *** Note that you probably will NOT want to include the author's ***
% *** name in the headers of peer review papers. ***
% You can use \ifCLASSOPTIONpeerreview for conditional compilation here if
% you desire.
% The publisher's ID mark at the bottom of the page is less important with
% Computer Society journal papers as those publications place the marks
% outside of the main text columns and, therefore, unlike regular IEEE
% journals, the available text space is not reduced by their presence.
% If you want to put a publisher's ID mark on the page you can do it like
% this:
%\IEEEpubid{0000--0000/00\$00.00~\copyright~2015 IEEE}
% or like this to get the Computer Society new two part style.
%\IEEEpubid{\makebox[\columnwidth]{\hfill 0000--0000/00/\$00.00~\copyright~2015 IEEE}%
%\hspace{\columnsep}\makebox[\columnwidth]{Published by the IEEE Computer Society\hfill}}
% Remember, if you use this you must call \IEEEpubidadjcol in the second
% column for its text to clear the IEEEpubid mark (Computer Society jorunal
% papers don't need this extra clearance.)
% use for special paper notices
%\IEEEspecialpapernotice{(Invited Paper)}
% for Computer Society papers, we must declare the abstract and index terms
% PRIOR to the title within the \IEEEtitleabstractindextext IEEEtran
% command as these need to go into the title area created by \maketitle.
% As a general rule, do not put math, special symbols or citations
% in the abstract or keywords.
\IEEEtitleabstractindextext{%
\input{sections/abstract.tex}
% Note that keywords are not normally used for peerreview papers.
\begin{IEEEkeywords}
Graph Convolution Network, Non-euclidean Data, 3D Semantic Segmentation, Node classification, Deep Learning
%Computer Society, IEEE, IEEEtran, journal, \LaTeX, paper, template.
\end{IEEEkeywords}}
% make the title area
\maketitle
% To allow for easy dual compilation without having to reenter the
% abstract/keywords data, the \IEEEtitleabstractindextext text will
% not be used in maketitle, but will appear (i.e., to be "transported")
% here as \IEEEdisplaynontitleabstractindextext when the compsoc
% or transmag modes are not selected <OR> if conference mode is selected
% - because all conference papers position the abstract like regular
% papers do.
\IEEEdisplaynontitleabstractindextext
% \IEEEdisplaynontitleabstractindextext has no effect when using
% compsoc or transmag under a non-conference mode.
% For peer review papers, you can put extra information on the cover
% page as needed:
% \ifCLASSOPTIONpeerreview
% \begin{center} \bfseries EDICS Category: 3-BBND \end{center}
% \fi
%
% For peerreview papers, this IEEEtran command inserts a page break and
% creates the second title. It will be ignored for other modes.
\IEEEpeerreviewmaketitle
%\MM{When a TPAMI submission is based on a previous conference paper, IEEE requires that the journal paper be a “substantial revision” of the previous publication (30 percent is generally considered “substantial”). TPAMI interprets and applies this requirement on a case-by-case basis with appropriate deference to the author’s viewpoint. Examples of the improvements we expect to see over the conference paper include the following: additional technical details, a clearer explanation of the contribution, more experiments if appropriate, or an updated state-of-the-art. Of course, the authors are also encouraged to make the journal version a significant improvement on the conference paper (for example, by taking the opportunity to integrate their previous work or performing additional substantive work to answer questions that their conference paper raised). Since the journal version is intended to be the definitive, archival version of the research, TPAMI expects that the authors will take this opportunity to further improve their conference paper.}
\input{sections/introduction.tex}
%\hfill mds
% \subsection{Subsection Heading Here}
% Subsection text here.
% % needed in second column of first page if using \IEEEpubid
% %\IEEEpubidadjcol
% \subsubsection{Subsubsection Heading Here}
% Subsubsection text here.
\input{sections/related_work.tex}
\input{sections/methodology.tex}
\input{sections/experiments.tex}
\input{sections/experiments2.tex}
% An example of a floating figure using the graphicx package.
% Note that \label must occur AFTER (or within) \caption.
% For figures, \caption should occur after the \includegraphics.
% Note that IEEEtran v1.7 and later has special internal code that
% is designed to preserve the operation of \label within \caption
% even when the captionsoff option is in effect. However, because
% of issues like this, it may be the safest practice to put all your
% \label just after \caption rather than within \caption{}.
%
% Reminder: the "draftcls" or "draftclsnofoot", not "draft", class
% option should be used if it is desired that the figures are to be
% displayed while in draft mode.
%
%\begin{figure}[!t]
%\centering
%\includegraphics[width=2.5in]{myfigure}
% where an .eps filename suffix will be assumed under latex,
% and a .pdf suffix will be assumed for pdflatex; or what has been declared
% via \DeclareGraphicsExtensions.
%\caption{Simulation results for the network.}
%\label{fig_sim}
%\end{figure}
% Note that the IEEE typically puts floats only at the top, even when this
% results in a large percentage of a column being occupied by floats.
% However, the Computer Society has been known to put floats at the bottom.
% An example of a double column floating figure using two subfigures.
% (The subfig.sty package must be loaded for this to work.)
% The subfigure \label commands are set within each subfloat command,
% and the \label for the overall figure must come after \caption.
% \hfil is used as a separator to get equal spacing.
% Watch out that the combined width of all the subfigures on a
% line do not exceed the text width or a line break will occur.
%
%\begin{figure*}[!t]
%\centering
%\subfloat[Case I]{\includegraphics[width=2.5in]{box}%
%\label{fig_first_case}}
%\hfil
%\subfloat[Case II]{\includegraphics[width=2.5in]{box}%
%\label{fig_second_case}}
%\caption{Simulation results for the network.}
%\label{fig_sim}
%\end{figure*}
%
% Note that often IEEE papers with subfigures do not employ subfigure
% captions (using the optional argument to \subfloat[]), but instead will
% reference/describe all of them (a), (b), etc., within the main caption.
% Be aware that for subfig.sty to generate the (a), (b), etc., subfigure
% labels, the optional argument to \subfloat must be present. If a
% subcaption is not desired, just leave its contents blank,
% e.g., \subfloat[].
% An example of a floating table. Note that, for IEEE style tables, the
% \caption command should come BEFORE the table and, given that table
% captions serve much like titles, are usually capitalized except for words
% such as a, an, and, as, at, but, by, for, in, nor, of, on, or, the, to
% and up, which are usually not capitalized unless they are the first or
% last word of the caption. Table text will default to \footnotesize as
% the IEEE normally uses this smaller font for tables.
% The \label must come after \caption as always.
%
%\begin{table}[!t]
%% increase table row spacing, adjust to taste
%\renewcommand{\arraystretch}{1.3}
% if using array.sty, it might be a good idea to tweak the value of
% \extrarowheight as needed to properly center the text within the cells
%\caption{An Example of a Table}
%\label{table_example}
%\centering
%% Some packages, such as MDW tools, offer better commands for making tables
%% than the plain LaTeX2e tabular which is used here.
%\begin{tabular}{|c||c|}
%\hline
%One & Two\\
%\hline
%Three & Four\\
%\hline
%\end{tabular}
%\end{table}
% Note that the IEEE does not put floats in the very first column
% - or typically anywhere on the first page for that matter. Also,
% in-text middle ("here") positioning is typically not used, but it
% is allowed and encouraged for Computer Society conferences (but
% not Computer Society journals). Most IEEE journals/conferences use
% top floats exclusively.
% Note that, LaTeX2e, unlike IEEE journals/conferences, places
% footnotes above bottom floats. This can be corrected via the
% \fnbelowfloat command of the stfloats package.
\input{sections/conclusion.tex}
% if have a single appendix:
%\appendix[Proof of the Zonklar Equations]
% or
%\appendix % for no appendix heading
% do not use \section anymore after \appendix, only \section*
% is possibly needed
% use appendices with more than one appendix
% then use \section to start each appendix
% you must declare a \section before using any
% \subsection or using \label (\appendices by itself
% starts a section numbered zero.)
%
%\input{sections/supplement.tex}
% \section{Proof of the First Zonklar Equation}
% Appendix one text goes here.
% % you can choose not to have a title for an appendix
% % if you want by leaving the argument blank
% \section{}
% Appendix two text goes here.
\input{sections/acknowledgement.tex}
% Can use something like this to put references on a page
% by themselves when using endfloat and the captionsoff option.
\ifCLASSOPTIONcaptionsoff
\newpage
\fi
% trigger a \newpage just before the given reference
% number - used to balance the columns on the last page
% adjust value as needed - may need to be readjusted if
% the document is modified later
%\IEEEtriggeratref{8}
% The "triggered" command can be changed if desired:
%\IEEEtriggercmd{\enlargethispage{-5in}}
% references section
% can use a bibliography generated by BibTeX as a .bbl file
% BibTeX documentation can be easily obtained at:
% http://mirror.ctan.org/biblio/bibtex/contrib/doc/
% The IEEEtran BibTeX style support page is at:
% http://www.michaelshell.org/tex/ieeetran/bibtex/
%\bibliographystyle{IEEEtran}
% argument is your BibTeX string definitions and bibliography database(s)
%\bibliography{IEEEabrv,../bib/paper}
%
% <OR> manually copy in the resultant .bbl file
% set second argument of \begin to the number of references
% (used to reserve space for the reference number labels box)
%\begin{thebibliography}{1}
\newpage
\bibliographystyle{IEEEtran}
\bibliography{references}
%\end{thebibliography}
% biography section
%
% If you have an EPS/PDF photo (graphicx package needed) extra braces are
% needed around the contents of the optional argument to biography to prevent
% the LaTeX parser from getting confused when it sees the complicated
% \includegraphics command within an optional argument. (You could create
% your own custom macro containing the \includegraphics command to make things
% simpler here.)
%\begin{IEEEbiography}[{\includegraphics[width=1in,height=1.25in,clip,keepaspectratio]{mshell}}]{Michael Shell}
% or if you just want to reserve a space for a photo:
\newpage
\input{sections/supplement}
\newpage
\begin{IEEEbiography}[{\includegraphics[width=1in,height=1.25in,clip,keepaspectratio]{portraits/GuohaoLI.png}}]{Guohao Li} obtained the BEng degree in Communication Engineering from Harbin Institute of Technology in 2015. In 2018, he received his Master Degree in Communication and Information Systems from Chinese Academy of Science. He was a research intern at SenseTime. He is currently a Computer Science PhD student at King Abdullah University of Science and Technology. His primary research interests are Computer Vision, Robotics and Deep Learning.
\end{IEEEbiography}
\begin{IEEEbiography}[{\includegraphics[width=1in,height=1.25in,clip,keepaspectratio]{portraits/matthias.jpg}}]{Matthias M{\"u}ller} received his PhD in computer vision from KAUST in 2019. He now works as a research scientist at the Intelligent Systems Lab at Intel. His research interests lie in the fields of computer vision, robotics and machine learning where he has contributed to more than 10 publications in top tier conferences and journals. Matthias was recognized as an outstanding reviewer for CVPR'18 and won the best paper award at the ECCV'18 workshop UAVision.
\end{IEEEbiography}
\begin{IEEEbiography}[{\includegraphics[width=1in,height=1.25in,clip,keepaspectratio]{portraits/qiang1in.jpg}}]{Guocheng Qian}
received the BEng degree with first class honors from Xi'an Jiaotong University, China in 2018. He is working towards the MSc degree currently in the Department of Computer Science at King Abdullah University of Science and Technology. His research interests include computer vision, computational photography and neural architecture search.
\end{IEEEbiography}
\begin{IEEEbiography}[{\includegraphics[width=1in,height=1.25in,clip,keepaspectratio]{portraits/itzel_carolina.jpg}}]{Itzel C. Delgadillo}
received the Bachelor in Artificial Intelligence from the Panamerican University, Mexico in 2019. She is currently a Visiting Student Research Intern in King Abdullah University of Science and Technology. Her research interests include computer vision and neural architecture search.
\end{IEEEbiography}
\begin{IEEEbiography}[{\includegraphics[width=1in,height=1.25in,clip,keepaspectratio]{portraits/abdulellah.jpg}}]{Abdulellah Abualshour}
received the BS degree in computer science from Rutgers, The State University of New Jersey in 2018. He was a recipient of the KAUST Gifted Student Program (KGSP) scholarship award. He is currently an MS student and a member of the Image and Video Understanding Lab (IVUL) at the Visual Computing Center (VCC) at King Abdullah University of Science and Technology (KAUST). His research interests include computer vision and deep learning.
\end{IEEEbiography}
\begin{IEEEbiography}[{\includegraphics[width=1in,height=1.25in,clip,keepaspectratio]{portraits/ali_thabet.jpg}}]{Ali Thabet}
is a Research Scientist at the Visual Computing Center (VCC) in King Abdullah University of Science and Technology (KAUST), working in the Image and Video Understanding Laboratory (IVUL). His research focuses on problems related to 3D computer vision. In general, he’s interested in algorithmic applications of machine learning, and deep learning specifically, to understand the 3D world with the help of sensors like RGB-D cameras, LiDAR, and others. Also, Ali is interested in applying deep learning to image reconstruction, 3D object detection and generation, and autonomous vehicles.
\end{IEEEbiography}
\begin{IEEEbiography}[{\includegraphics[width=1in,height=1.25in,clip,keepaspectratio]{portraits/Bernard_Ghanem.JPG}}]{Bernard Ghanem}
is currently an associate professor with the King Abdullah University of Science and Technology (KAUST), in the Visual Computing Center (VCC). He leads the Image and Video Understanding Lab (IVUL), KAUST. He is involved in several interesting projects that focus on exploiting techniques in computer vision and machine learning for real-world applications including semantic sports video analysis, large-scale activity recognition/detection, and real-time crowd analysis. He has published more than 100 peer-papers in peer-reviewed venues including the IEEE Transactions on Pattern Analysis and Machine Intelligence, the International Journal of Computer Vision, CVPR, ICCV, ECCV, etc. He is a member of the IEEE.
\end{IEEEbiography}
% You can push biographies down or up by placing
% a \vfill before or after them. The appropriate
% use of \vfill depends on what kind of text is
% on the last page and whether or not the columns
% are being equalized.
%\vfill
% Can be used to pull up biographies so that the bottom of the last one
% is flush with the other column.
%\enlargethispage{-5in}
% that's all folks
\end{document}
%%%%%%%%% ABSTRACT
\begin{abstract}
Convolutional Neural Networks (CNNs) have been very successful at solving a variety of computer vision tasks such as object classification and detection, semantic segmentation, activity understanding, to name just a few. One key enabling factor for their great performance has been the ability to train very deep networks. Despite their huge success in many tasks, CNNs do not work well with non-Euclidean data, which is prevalent in many real-world applications. Graph Convolutional Networks (GCNs) offer an alternative that allows for non-Eucledian data input to a neural network. While GCNs already achieve encouraging results, they are currently limited to architectures with a relatively small number of layers, primarily due to vanishing gradients during training. This work transfers concepts such as residual/dense connections and dilated convolutions from CNNs to GCNs in order to successfully train very deep GCNs. We show the benefit of using deep GCNs (with as many as $112$ layers) experimentally across various datasets and tasks. Specifically, we achieve state-of-the-art performance in part segmentation and semantic segmentation on point clouds and in node classification of protein functions across biological protein-protein interaction (PPI) graphs. We believe that the insights in this work will open avenues for future research on GCNs and their application to further tasks not explored in this paper. The source code for this work is available at \href{https://github.com/lightaime/deep_gcns_torch}{https://github.com/lightaime/deep\_gcns\_torch} and \href{https://github.com/lightaime/deep_gcns}{https://github.com/lightaime/deep\_gcns} for PyTorch and TensorFlow implementation respectively.
\end{abstract}
%%%%%%%%% BODY TEXT
%\vspace{-8pt}
\IEEEraisesectionheading{\section{Introduction}\label{sec:introduction}}
% Computer Society journal (but not conference!) papers do something unusual
% with the very first section heading (almost always called "Introduction").
% They place it ABOVE the main text! IEEEtran.cls does not automatically do
% this for you, but you can achieve this effect with the provided
% \IEEEraisesectionheading{} command. Note the need to keep any \label that
% is to refer to the section immediately after \section in the above as
% \IEEEraisesectionheading puts \section within a raised box.
% The very first letter is a 2 line initial drop letter followed
% by the rest of the first word in caps (small caps for compsoc).
%
% form to use if the first word consists of a single letter:
% \IEEEPARstart{A}{demo} file is ....
%
% form to use if you need the single drop letter followed by
% normal text (unknown if ever used by the IEEE):
% \IEEEPARstart{A}{}demo file is ....
%
% Some journals put the first two words in caps:
% \IEEEPARstart{T}{his demo} file is ....
%
% Here we have the typical use of a "T" for an initial drop letter
% and "HIS" in caps to complete the first word.
%\hfill October 1, 2019
\IEEEPARstart{G}{CNs} have become a prominent research topic in recent years. There are several reasons for this trend, but above all, GCNs promise a natural extension of CNNs to non-Euclidean data. While CNNs are very powerful when dealing with grid-like structured data, \eg images, their performance on more irregular data, \eg point clouds, graphs, \etc, is sub-par. Since many real-world applications need to leverage such data, GCNs are a very natural fit. There has already been some success in using GCNs to predict individual relations in social networks \cite{social_tang2009relational}, model proteins for drug discovery \cite{chem_zitnik2017predicting,chem_wale2008comparison}, enhance predictions of recommendation engines \cite{rec_monti2017geometric,rec_ying2018graph}, and efficiently segment large point clouds \cite{wang2018dynamic}. While these works show promising results, they rely on simple and shallow network architectures.
In the case of CNNs, the primary reason for their continued success and state-of-the-art performance on many computer vision tasks, is the ability to reliably train very deep network architectures. Surprisingly, it is not clear how to train deep GCN architectures and many existing works have investigated this limitation along with other shortcomings of GCNs \cite{li2018deeper,wu2019comprehensive,zhou2018graph}. Similar to CNNs, stacking multiple layers in GCNs leads to the vanishing gradient problem.
The over-smoothing problem can also occurs when repeatedly applying many GCN layers \cite{li2018deeper}. In this case, it was observed that the features of vertices within each connected component will converge to the same value and thus become indistinguishable from each other. As a result, most state-of-the-art GCNs are limited to shallow network architectures, usually no deeper than $4$ layers \cite{zhou2018graph}.
\begin{figure}[!th]
\centering
\begin{tabular}{cc}
\includegraphics[trim=5mm 0mm 0mm 0mm, width=1\columnwidth]{figures/intro_fig6.pdf}
\end{tabular}
\caption{\textbf{Training DeepGCNs}. (\emph{left}) We show the square root of the training loss for GCNs with 7, 14, 28, 56 and 112 layers, with and without residual connections for $100$ epochs. We note that adding more layers without residual connections translates to a substantially higher loss and for very deep networks (\eg 112 layers) even to divergence. (\emph{right}) In contrast, training GCNs with residual connections results in consistent training stability across all depths. \R{All training losses are obtained by training ResGCNs with varying depth but the same hyperparameters for the task of semantic segmentation on the S3DIS dataset.}}
\label{fig:intro_fig}
\end{figure}
The vanishing gradient problem is well-known and well-studied in the realm of CNNs. As a matter of fact, it was the key limitation for deep convolutional networks before ResNet \cite{he2016deep} proposed a simple, yet effective solution. The introduction of residual connections \cite{he2016deep} between consecutive layers addressed the vanishing gradient problem by providing additional paths for the gradient.
This enabled deep residual networks with more than a hundred layers (\eg ResNet-152) to be trained reliably. The idea was further extended by DenseNet \cite{huang2017densely}, where additional connections are added across layers.
Training deep networks reveals another bottleneck, which is especially relevant for tasks that rely on the spatial composition of the input image, \eg object detection, semantic segmentation, depth estimation, \etc With increased network depth, more spatial information can potentially be lost during pooling. Ideally, the receptive field should increase with network depth without loss of resolution. For CNNs, dilated convolutions \cite{yu2015multi} were introduced to tackle this issue. The idea is again simple but effective. Essentially, the convolutions are performed across more distant neighbors as the network depth increases. In this way, multiple resolutions can be seamlessly encoded in deeper CNNs. Several innovations, in particular residual/dense connections and dilated convolutions, have enabled reliable training of very deep CNNs achieving state-of-the-art performance on many tasks. This yields the following question: do these innovations have a counterpart in the realm of GCNs?
In this work, we present an extensive study of methodologies that allow training very deep GCNs. We adapt concepts that were successful in training deep CNNs, in particular residual connections, dense connections, and dilated convolutions. We show how these concepts can be incorporated into a graph framework. In order to quantify the effect of these additions, we conduct an extensive analysis of each component and its impact on accuracy and stability of deep GCNs. To showcase the potential of these concepts in the context of GCNs, we apply them to the popular tasks of semantic segmentation and part segmentation of point clouds as well as node classification of biological graphs. Adding either residual or dense connections in combination with dilated convolutions, enables successful training of GCNs with a depth of $112$ layers (refer to \figLabel \ref{fig:intro_fig}). The proposed deep GCNs improve the state-of-the-art on the challenging point cloud dataset S3DIS \cite{2017arXiv170201105A} by $3.9\%$ mIOU and outperform previous methods in many classes of PartNet \cite{Mo_2019_CVPR}. The same deep GCN architecture achieves an F1 score of $99.43$ on the very different PPI dataset \cite{chem_zitnik2017predicting}.
\vspace{6pt}\mysection{Contributions} The contributions of this work are three fold. \textbf{(1)} We adapt residual connections, dense connections, and dilated convolutions, which were introduced for CNNs to enable deep GCN architectures, denoted DeepGCNs. \textbf{(2)} We present extensive experiments on point cloud and biological graph data, showing the effect of each component to the stability and performance of training deep GCNs. We use semantic segmentation and part segmentation on point clouds, as well as, node classification of biological networks as our experimental testbeds. \textbf{(3)} We show how these new concepts enable successful training of a $112$-layer GCN, the deepest GCN architecture by a large margin. With only 28 layers, we already improve the previous best performance by almost $4\%$ in terms of mIOU on the S3DIS dataset \cite{2017arXiv170201105A}; we also outperform previous methods in the task of part segmentation for many classes of PartNet \cite{Mo_2019_CVPR}. Similarly, we set a new state-of-art for the PPI dataset \cite{chem_zitnik2017predicting} in the task of node classification in biological networks.
A preliminary version of this work was published in \cite{li2019deepgcns}. This journal manuscript extends the initial version in several aspects. \emph{First}, we investigate even deeper \emph{DeepGCN} architectures with more than 100 layers. Interestingly, we find that divergence occurs when training a \emph{PlainGCN} with 112 layers, while our proposed counterpart, \emph{ResGCN} with skip connections and dilated convolutions, converges without problem (see \figLabel \ref{fig:intro_fig}).
\emph{Second}, to investigate the generality of our \emph{DeepGCN} framework, we perform extensive additional experiments on the tasks of part segmentation on PartNet and node classification on PPI. \emph{Third}, we examine the performance and efficiency of \emph{MRGCN}, a memory-efficient GCN aggregator we propose, with thorough experiments on the PPI dataset. Our results show that \emph{DeepMRGCN} models are able to outperform current state-of-the-art methods. We also demonstrate that \emph{MRGCN} is very memory-efficient compared to other GCN operators via GPU memory usage experiments. \emph{Finally}, to ensure the reproducibility of our experiments and contribute to the graph learning research community, we have published code for training, testing, and visualization along with several pretrained models in both TensorFlow and PyTorch. To the best of our knowledge, our work is the first to successfully train deep GCNs beyond 100 layers and achieves state-of-the-art results on both point cloud and biological graph data.
\section{Related Work}
\label{sec:related}
A large number of real-world applications deal with non-Euclidean data, which cannot be systematically and reliably processed by CNNs in general. To overcome the shortcomings of CNNs, GCNs provide well-suited solutions for non-Euclidean data processing, leading to greatly increasing interest in using GCNs for a variety of applications. In social networks \cite{social_tang2009relational}, graphs represent connections between individuals based on mutual interests/relations. These connections are non-Euclidean and highly irregular. GCNs help better estimate edge strengths between the vertices of social network graphs, thus leading to more accurate connections between individuals. Graphs are also used to model chemical molecule structures \cite{chem_zitnik2017predicting,chem_wale2008comparison}. Understanding the bio-activities of these molecules can have substantial impact on drug discovery. Another popular use of graphs is in recommendation engines \cite{rec_monti2017geometric,rec_ying2018graph}, where accurate modelling of user interactions leads to improved product recommendations. Graphs are also popular modes of representation in natural language processing \cite{nlp_bastings2017graph,nlp_marcheggiani2017encoding}, where they are used to represent complex relations between large text units.
% words, sentences, and larger text units.
GCNs also find many applications in computer vision. In scene graph generation, semantic relations between objects are modelled using a graph. This graph is used to detect and segment objects in images, and also to predict semantic relations between object pairs \cite{qi20173d,cv_scene_xu2017scene,cv_scene_yang2018graph,cv_scene_li2018factorizable}. Scene graphs facilitate the inverse process as well, where an image is reconstructed given a graph representation of the scene \cite{cv_inv_scene_johnson2018image}. Graphs are also used to model human joints for action recognition in video \cite{cv_action_yan2018spatial,cv_action_jain2016structural}.
Moreover, GCNs are a perfect candidate for 3D point cloud processing, especially since the unstructured nature of point clouds poses a representational challenge for systematic research. Several attempts in creating structure from 3D data exist by either representing it with multiple 2D views \cite{mv_su2015multi,mv_guerry2017snapnet,mv_boulch2017unstructured,mv_li2016lstm}, or by voxelization \cite{voxel_dai2017scannet,voxel_mv_qi2016volumetric,voxel_riegler2017octnet,voxel_tchapmi2017segcloud}. More recent work focuses on directly processing unordered point cloud representations \cite{pc_qi2017pointnet,pc_qi2017pointnet++, 3dsemseg_ICCVW17,pc_huang2018recurrent,pc_ye20183d}. The recent \emph{EdgeConv} method by Wang \etal \cite{wang2018dynamic} applies GCNs to point clouds. In particular, they propose a dynamic edge convolution algorithm for semantic segmentation of point clouds. The algorithm dynamically computes node adjacency at each graph layer using the distance between point features. This work demonstrates the potential of GCNs for point cloud related applications and beats the state-of-the-art in the task of point cloud segmentation. Unlike most other works, \emph{EdgeConv} does not rely on RNNs or complex point aggregation methods.
Current GCN algorithms including \emph{EdgeConv} are limited to shallow depths. Recent works have attempted to train deeper GCNs. For instance, Kipf \etal trained a semi-supervised GCN model for node classification and showed how performance degrades when using more than 3 layers \cite{kipf2016semi}. Pham \etal \cite{pham2017column} proposed Column Network (CLN) for collective classification in relational learning, where they showed peak performance at 10 layers and degrading performance for deeper graphs. Rahimi \etal \cite{rahimi2018semi} developed a Highway GCN for user geo-location in social media graphs, where they add ``highway" gates between layers to facilitate gradient flow. Even with these gates, the authors demonstrate performance degradation after 6 layers of depth. Xu \etal \cite{xu2018representation} developed a \emph{Jump Knowledge Network} for representation learning and devised an alternative strategy to select graph neighbors for each node based on graph structure. As with other works, their network is limited to a small number of layers ($6$). Recently, Li \etal \cite{li2018deeper} studied the depth limitations of GCNs and showed that deep GCNs can cause over-smoothing, which results in features at vertices within each connected component converging to the same value. Other works \cite{wu2019comprehensive,zhou2018graph} also show the limitations of stacking GCN layers, specifically highly complex back-propagation and the common vanishing gradient problem.
Many difficulties facing GCNs nowadays (\eg vanishing gradients and limited receptive field) were also present in the early days of CNNs \cite{he2016deep,yu2015multi}. We bridge this gap and show that the majority of these drawbacks can be remedied by borrowing several orthogonal tricks from CNNs. Deep CNNs achieved a huge boost in performance with the introduction of ResNet \cite{he2016deep}. By adding residual connections between inputs and outputs of layers, ResNet tends to alleviate the vanishing gradient problem. DenseNet \cite{huang2017densely} takes this idea a step further and adds connections across layers as well. Dilated Convolutions \cite{yu2015multi} are another recent approach that has lead to significant performance gains, specifically in image-to-image translation tasks such as semantic segmentation \cite{yu2015multi}, by increasing the receptive field without loss of resolution. In this work, we show how one can benefit from concepts introduced for CNNs, mainly residual/dense connections and dilated convolutions, to train very deep GCNs. We support our claim by extending %the work of Wang \etal \cite{wang2018dynamic} to a much deeper GCN
different GCN variants to deeper versions through adapting these concepts, and therefore significantly increasing their performance. Extensive experiments on the tasks of semantic segmentation and part segmentation of point clouds and node classification in biological graphs validate these ideas for general graph scenarios.
\section{Methodology}
\label{sec:methodology}
\begin{figure*}
\centering
\includegraphics[page=2,trim = 10mm 10mm 40mm 10mm, clip, width=\textwidth]{figures/Pipeline.pdf}
\caption{\textbf{Proposed GCN architecture for point cloud semantic segmentation}. \textit{(left)} Our framework consists of three blocks: a GCN Backbone Block (feature transformation of input point cloud), a Fusion Block (global feature generation and fusion), and an MLP Prediction Block (point-wise label prediction). \textit{(right)} We study three types of GCN Backbone Block (\emph{PlainGCN}, \emph{ResGCN} and \emph{DenseGCN}) and use two kinds of layer connection (vertex-wise addition used in \emph{ResGCN} or vertex-wise concatenation used in \emph{DenseGCN}).
}
\label{fig:pipeline}
\end{figure*}
\subsection{Representation Learning on Graphs} \label{sec:defineGCN}
\mysection{Graph Definition} A graph $\mathcal{G}$ is represented by a tuple $\mathcal{G}=(\mathcal{V}, \mathcal{E})$ where $\mathcal{V}$ is the set of unordered vertices and $\mathcal{E}$ is the set of edges representing the connectivity between vertices $v \in \mathcal{V}$. If $e_{i,j} \in \mathcal{E}$, then vertices $v_i$ and $v_j$ are connected to each other with an edge $e_{i,j}$.
\mysection{Graph Convolution Networks} Inspired by CNNs, GCNs intend to extract richer features at a vertex by aggregating features of vertices from its neighborhood. GCNs represent vertices by associating each vertex $v$ with a feature vector $\mathbf{h}_{v} \in \mathbb{R}^{D}$,
where $D$ is the feature dimension. Therefore, the graph $\mathcal{G}$ as a whole can be represented by concatenating the features of all the unordered vertices, \ie $\mathbf{h}_{\mathcal{G}} = [\mathbf{h}_{v_1}, \mathbf{h}_{v_2}, ..., \mathbf{h}_{v_N}]^{\top} \in \mathbb{R}^{N\times D}$,
where $N$ is the cardinality of set $\mathcal{V}$. A general graph convolution operation $\mathcal{F}$ at the $l$-th layer can be formulated as the following aggregation and update operations,
\begin{equation} \label{eq:GCN}
\begin{split}
\mathcal{G}_{l+1} &=\mathcal{F}(\mathcal{G}_{l}, \mathcal{W}_l)\\
&= Update(Aggregate(\mathcal{G}_{l}, \mathcal{W}_l^{agg}), \mathcal{W}_l^{update}).
\end{split}
\end{equation}
$\mathcal{G}_l = (\mathcal{V}_{l}, \mathcal{E}_{l})$ and $\mathcal{G}_{l+1} = (\mathcal{V}_{l+1}, \mathcal{E}_{l+1})$ are the input and output graphs at the $l$-th layer, respectively. $\mathcal{W}_l^{agg}$ and $\mathcal{W}_l^{update}$ are the learnable weights of the aggregation and update functions respectively, and they are the essential components of GCNs. In most GCN frameworks, aggregation functions are used to compile information from the neighborhood of vertices, while update functions perform a non-linear transform on the aggregated information to compute new vertex representations. There are different variants of these two functions. For example, the aggregation function can be a mean aggregator \cite{kipf2016semi}, a max-pooling aggregator \cite{pc_qi2017pointnet,hamilton2017inductive,wang2018dynamic}, an attention aggregator \cite{velivckovic2017graph}, or an LSTM aggregator \cite{peng2017cross}. The update function can be a multi-layer perceptron \cite{hamilton2017inductive,duvenaud2015convolutional}, a gated network \cite{li2015gated}, \etc More concretely, the representation of vertices is computed at each layer by aggregating features of neighbor vertices for all $v_{l+1} \in \mathcal{V}_{l+1} $ as follows,
\begin{equation} \label{GCN_node}
\resizebox{.89\columnwidth}{!}
{$\mathbf{h}_{v_{l+1}} =\phi\left(\mathbf{h}_{v_{l}}, \rho(\{\mathbf{h}_{u_{l}}| u_{l}\in \mathcal{N}(v_{l})\}, \mathbf{h}_{v_{l}}, \mathcal{W}_{\rho}), \mathcal{W}_{\phi}\right)$},
\end{equation}
where $\rho$ is a vertex feature aggregation function and $\phi$ is a vertex feature update function, $\mathbf{h}_{v_{l}}$ and $\mathbf{h}_{v_{l+1}}$ are the vertex features at the $l$-th layer and $(l+1)$-th layer, respectively. $\mathcal{N}(v_{l})$ is the set of neighbor vertices of $v$ at the $l$-th layer, and $\mathbf{h}_{u_l}$ is the feature of those neighbor vertices parametrized by $\mathcal{W}_{\rho}$. $\mathcal{W}_{\phi}$ contains the learnable parameters of these functions. For simplicity and without loss of generality, we use a max-pooling vertex feature aggregator, without learnable parameters, to pool the difference of features between vertex $v_l$ and all of its neighbors: $\rho(.)=\max(\mathbf{h}_{u_{l}} - \mathbf{h}_{v_{l}}|~ u_{l}\in \mathcal{N}(v_{l}))$. We then model the vertex feature updater $\phi$ as a multi-layer perceptron (MLP) with batch normalization \cite{ioffe2015batch} and a ReLU as an activation function. This MLP concatenates $\mathbf{h}_{v_{l}}$ with its aggregate features from $\rho(.)$ to form its input.
\mysection{Dynamic Edges} \label{dyna} As mentioned earlier, most GCNs have fixed graph structures and only update the vertex features at each iteration. Recent work \cite{simonovsky2017dynamic,wang2018dynamic,valsesia2018learning} demonstrates that dynamic graph convolution, where the graph structure is allowed to change in each layer, can learn better graph representations compared to GCNs with fixed graph structure. For instance, ECC (Edge-Conditioned
Convolution) \cite{simonovsky2017dynamic} uses dynamic edge-conditional filters to learn an edge-specific weight matrix. Moreover, EdgeConv \cite{wang2018dynamic} finds the nearest neighbors in the current feature space to reconstruct the graph after every EdgeConv layer. In order to learn to generate point clouds, Graph-Convolution GAN (Generative Adversarial Network) \cite{valsesia2018learning} also applies $k$-NN graphs to construct the neighbourhood of each vertex in every layer. We find that dynamically changing neighbors in GCNs
results in an effectively larger receptive field, when deeper GCNs are considered. In our framework, we propose to re-compute edges between vertices via a \emph{Dilated $k$-NN} function in the feature space of each layer to further increase the receptive field.
Designing deep GCN architectures \cite{wu2019comprehensive, zhou2018graph} is an open problem in the graph learning domain. Recent work \cite{li2018deeper, wu2019comprehensive, zhou2018graph} suggests that GCNs do not scale well to deep architectures, since stacking multiple layers of graph convolutions leads to high complexity in back-propagation. As such, most state-of-the-art GCN models are usually quite shallow \cite{zhou2018graph}. Inspired by the huge success of ResNet \cite{he2016deep}, DenseNet \cite{huang2017densely}, and Dilated Convolutions \cite{yu2015multi}, we transfer these ideas to GCNs to unleash their full potential. This enables much deeper GCNs that reliably converge in training and achieve superior performance in inference. In what follows, we provide a detailed description of three operations that can enable much deeper GCNs: residual connections, dense connections, and dilated aggregation.
\subsection{Residual Connections for GCNs} \label{sec:ResGCN}
In the original graph learning framework, the underlying mapping $\mathcal{F}$, which takes a graph as an input and outputs a new graph representation (see \eqLabel \eqref{eq:GCN}), is learned.
Here, we propose a graph residual learning framework that learns an underlying mapping $\mathcal{H}$ by fitting another mapping $\mathcal{F}$. After $\mathcal{G}_{l}$ is transformed by $\mathcal{F}$, vertex-wise addition is performed to obtain $\mathcal{G}_{l+1}$. The residual mapping $\mathcal{F}$ learns to take a graph as input and outputs a residual graph representation $\mathcal{G}_{l+1}^{res}$ for the next layer. $\mathcal{W}_l$ is the set of learnable parameters at layer $l$. In our experiments, we refer to our residual model as \emph{ResGCN}.
\begin{equation} \label{eq2}
\begin{split}
\mathcal{G}_{l+1} &= \mathcal{H}(\mathcal{G}_{l}, \mathcal{W}_l) \\
&= \mathcal{F}(\mathcal{G}_{l}, \mathcal{W}_l) + \mathcal{G}_{l}=\mathcal{G}_{l+1}^{res}+ \mathcal{G}_{l}.
\end{split}
\end{equation}
\mysection{Dense Connections for GCNs} \label{sec:DenseGCN}
DenseNet \cite{huang2017densely} was proposed to exploit dense connectivity among layers, which improves information flow in the network and enables efficient reuse of features among layers. Inspired by DenseNet, we adapt a similar idea to GCNs so as to exploit information flow from different GCN layers. In particular, we have:
\begin{equation} \label{eq4}
\begin{split}
\mathcal{G}_{l+1} &= \mathcal{H}(\mathcal{G}_{l}, \mathcal{W}_l) \\
&= \mathcal{T}(\mathcal{F}(\mathcal{G}_{l}, \mathcal{W}_l), \mathcal{G}_{l}) \\
&= \mathcal{T}(\mathcal{F}(\mathcal{G}_{l}, \mathcal{W}_l), ...,\mathcal{F}(\mathcal{G}_{0}, \mathcal{W}_{0}), \mathcal{G}_{0}).
\end{split}
\end{equation}
The operator $\mathcal{T}$ is a vertex-wise concatenation function that densely fuses the input graph $\mathcal{G}_0$ with all the intermediate GCN layer outputs. To this end, $\mathcal{G}_{l+1}$ consists of all the GCN transitions from previous layers. Since we fuse GCN representations densely, we refer to our dense model as \emph{DenseGCN}. The growth rate of \emph{DenseGCN} is equal to the dimension $D$ of the output graph (similar to DenseNet for CNNs \cite{huang2017densely}). For example, if $\mathcal{F}$ produces a $D$ dimensional vertex feature, where the vertices of the input graph $\mathcal{G}_0$ are $D_{0}$ dimensional, the dimension of each vertex feature of $\mathcal{G}_{l+1}$ is $D_{0}+D\times (l+1)$.
\subsection{Dilated Aggregation for GCNs} \label{sec:dilation}
Dilated wavelet convolution is an algorithm originating from the wavelet processing domain \cite{holschneider1990real,shensa1992discrete}. To alleviate spatial information loss caused by pooling operations, Yu \etal \cite{yu2015multi} propose dilated convolutions as an alternative to applying consecutive pooling layers for dense prediction tasks, \eg semantic image segmentation. Their experiments demonstrate that aggregating multi-scale contextual information using dilated convolutions can significantly increase the accuracy on the semantic segmentation task. The reason behind this is the fact that dilation enlarges the receptive field without loss of resolution. We believe that dilation can also help with the receptive field of DeepGCNs.
Therefore, we introduce dilated aggregation to GCNs. There are many possible ways to construct a dilated neighborhood. We use a \emph{Dilated $k$-NN} to find dilated neighbors after every GCN layer and construct a \emph{Dilated Graph}. In particular, for an input graph $\mathcal{G}=(\mathcal{V}, \mathcal{E})$ with \emph{Dilated $k$-NN} and $d$ as the dilation rate, the \emph{Dilated $k$-NN} operation returns the $k$ nearest neighbors within the $k\times d$ neighborhood region by skipping every $d$ neighbors. The nearest neighbors are determined based on a pre-defined distance metric. In our experiments, we use the $\ell_2$ distance in the feature space of the current layer.
Let $\mathcal{N}^{(d)}(v)$ denote the $d$-dilated neighborhood of vertex $v$. If $(u_1, u_2, ..., u_{k\times d})$ are the first sorted $k\times d$ nearest neighbors, vertices $(u_1, u_{1+d}, u_{1+2d}, ..., u_{1+(k-1)d})$ are the $d$-dilated neighbors of vertex $v$ (see \figLabel \ref{fig:dilation_viz}), \ie
\begin{equation}\label{eq5}
\mathcal{N}^{(d)}(v) = \{u_1, u_{1+d}, u_{1+2d}, ..., u_{1+(k-1)d}\}.
\end{equation}
\begin{figure}[!htb]
\centering
\includegraphics[page=2,trim = 25mm 10mm 30mm 1mm, clip, width=\columnwidth]{figures/Dilation.pdf}
\caption{\textbf{Dilated Convolution in GCNs}. Visualization of dilated convolution on a structured graph arranged in a grid (\eg 2D image) and on a general structured graph. (\emph{top}) 2D convolution with kernel size 3 and dilation rate 1, 2, 4 (left to right). (\emph{bottom}) Dynamic graph convolution with dilation rate 1, 2, 4 (left to right).}
%\vspace{-6pt}
\label{fig:dilation_viz}
\end{figure}
Hence, the edges $\mathcal{E}^{(d)}$ of the output graph are defined on the set of $d$-dilated vertex neighbors $\mathcal{N}^{(d)}(v)$. Specifically, there exists a directed edge $e \in \mathcal{E}^{(d)}$ from vertex $v$ to every vertex $u \in \mathcal{N}^{(d)}(v)$. The GCN aggregation and update functions are applied, as in \eqLabel \eqref{eq:GCN}, by using the edges $\mathcal{E}^{(d)}$ created by the \emph{Dilated $k$-NN}, so as to generate the feature $\mathbf{h}_{v}^{(d)}$ of each output vertex in $\mathcal{V}^{(d)}$. We denote this layer operation as a \emph{dilated graph convolution} with dilation rate $d$, or more formally: $\mathcal{G}^{(d)}=(\mathcal{V}^{(d)}, \mathcal{E}^{(d)})$. We visualize and compare it %the proposed \emph{dilated graph convolution}
to a conventional dilated convolution used in CNNs in \figLabel \ref{fig:dilation_viz}. To improve generalization, we use \emph{stochastic dilation} in practice. During training, we perform the aforementioned dilated aggregations with a high probability $(1-\epsilon)$ leaving a small probability $\epsilon$ to perform random aggregation by uniformly sampling $k$ neighbors from the set of $k\times d$ neighbors $\{u_1, u_2, ..., u_{k\times d}\}$. At inference time, we perform deterministic dilated aggregation without stochasticity, \R{\ie the dilated neighbors are sampled based on \eqLabel \ref{eq5} with probability $1$}.
\subsection{Deep GCN Variants} \label{appendix:gcn_variants}
In our experiments in the paper, we mostly work with a GCN based on EdgeConv \cite{wang2018dynamic} to show how very deep GCNs can be trained. However, it is straightforward to build other deep GCNs with the same aforementioned concepts (\eg residual/dense graph connections and dilated graph convolutions). To show that these concepts are universal operators and can be used for general GCNs, we perform additional experiments. In particular, we build ResGCNs based on GraphSAGE \cite{hamilton2017inductive} and Graph Isomorphism Network (GIN) \cite{xu2018powerful}. In practice, we find that EdgeConv learns a better representation than the other implementations. However, it is less efficient in terms of memory and computation. Therefore, we also propose a simple GCN operation combining the advantages of both, which we refer to as \emph{MRGCN} (Max-Relative GCN). In the following, we discuss each GCN operator in detail.
\mysection{EdgeConv} Instead of aggregating neighborhood features directly, EdgeConv \cite{wang2018dynamic} proposes to first get local neighborhood information for each neighbor by subtracting the feature of the central vertex from its own feature. In order to train deeper GCNs, we add residual/dense graph connections and dilated graph convolutions to EdgeConv:
\begin{equation}
\begin{split}
\mathbf{h}^{res}_{v_{l+1}} &=\textit{max}\left(\{\textit{mlp}(\textit{concat}(\mathbf{h}_{v_{l}}, \mathbf{h}_{u_{l}}-\mathbf{h}_{v_{l}}))|u_{l}\in \mathcal{N}^{(d)}(v_{l})\}\right), \\
\mathbf{h}_{v_{l+1}} &= \mathbf{h}^{res}_{v_{l+1}} + \mathbf{h}_{v_{l}}.
\end{split}
\end{equation}
\mysection{GraphSAGE} GraphSAGE \cite{hamilton2017inductive} proposes different types of aggregator functions including a Mean aggregator, an LSTM aggregator, and a Pooling aggregator. Their experiments show that the Pooling aggregator outperforms the others. We adapt GraphSAGE with the max-pooling aggregator to obtain \emph{ResGraphSAGE}:
\begin{equation} \label{eq:sage}
\begin{split}
\mathbf{h}^{res}_{\mathcal{N}^{(d)}(v_{l})} &=\textit{max}\left(\{\textit{mlp}(\mathbf{h}_{u_{l}})|u_{l}\in \mathcal{N}^{(d)}(v_{l})\}\right), \\
\mathbf{h}^{res}_{v_{l+1}} &=\textit{mlp}\left(\textit{concat}\bigg(\mathbf{h}_{v_{l}}, \mathbf{h}^{res}_{\mathcal{N}^{(d)}(v_{l})}\bigg)\right), \\
\mathbf{h}_{v_{l+1}} &= \mathbf{h}^{res}_{v_{l+1}} + \mathbf{h}_{v_{l}},
\end{split}
\end{equation}
In the original GraphSAGE paper, the vertex features are normalized after aggregation. We implement two variants, one without normalization (see \eqLabel \eqref{eq:sage}), another with normalization $\mathbf{h}^{res}_{v_{l+1}} = \mathbf{h}^{res}_{v_{l+1}} / \left\Vert \mathbf{h}^{res}_{v_{l+1}} \right\Vert_{2}$.
\mysection{GIN} The main difference between GIN \cite{xu2018powerful} and other GCNs is that an $\epsilon$ is learned at each GCN layer to give the central vertex and aggregated neighborhood features different weights. Hence \emph{ResGIN} is formulated as follows:
\begin{equation}
\begin{split}
\mathbf{h}^{res}_{v_{l+1}} &=\textit{mlp}\left((1+\epsilon)\cdot \mathbf{h}_{v_{l}} + \textit{sum}(\{\mathbf{h}_{u_{l}}|u_{l}\in \mathcal{N}^{(d)}(v_{l})\})\right), \\
\mathbf{h}_{v_{l+1}} &= \mathbf{h}^{res}_{v_{l+1}} + \mathbf{h}_{v_{l}}.
\end{split}
\end{equation}
\mysection{MRGCN} We find that first using a max aggregator to aggregate neighborhood relative features $(\mathbf{h}_{u_{l}} - \mathbf{h}_{v_{l}}) ,~ u_{l}\in \mathcal{N}(v_{l})$ is more effective and efficient than aggregating raw neighborhood features $\mathbf{h}_{v_{l}} ,~ u_{l}\in \mathcal{N}(v_{l})$ or aggregating features after non-linear transforms. We refer to this simple GCN as \emph{MRGCN} (Max-Relative GCN). The residual version of \emph{MRGCN} is as such:
\begin{equation}
\begin{split}
\mathbf{h}^{res}_{\mathcal{N}^{(d)}(v_{l})} &=\textit{max}\left(\{\mathbf{h}_{u_{l}}-\mathbf{h}_{v_{l}}|u_{l}\in \mathcal{N}^{(d)}(v_{l})\}\right), \\
\mathbf{h}^{res}_{v_{l+1}} &=\textit{mlp}\left(\textit{concat}\bigg(\mathbf{h}_{v_{l}}, \mathbf{h}^{res}_{\mathcal{N}^{(d)}(v_{l})}\bigg)\right), \\
\mathbf{h}_{v_{l+1}} &= \mathbf{h}^{res}_{v_{l+1}} + \mathbf{h}_{v_{l}}.
\end{split}
\end{equation}
Here, $\mathbf{h}_{v_{l+1}}$ and $\mathbf{h}_{v_{l}}$ are the hidden states of vertex $v$ at layers $l$ and $l+1$, and $\mathbf{h}^{res}_{v_{l+1}}$ is the hidden state of the residual graph. All the \emph{mlp} (multilayer perceptron) functions use a ReLU as activation function; all the \emph{max} and \emph{sum} functions above are vertex-wise feature operators; \emph{concat} functions concatenate features of two vertices into one feature vector. $\mathcal{N}^{(d)}(v_{l})$ denotes the neighborhood of vertex $v_l$ obtained from \emph{Dilated $k$-NN}. \R{\emph{MRGCN} is more efficient than \emph{EdgeConv} and \emph{GraphSAGE}. \emph{MRGCN} only applies an \emph{mlp} transform once for each vertex, while \emph{EdgeConv} needs to apply this transform for every neighbor of each vertex. In contrast to \emph{GraphSAGE}, \emph{MRGCN} does not need to apply \emph{mlp} transforms to the neighborhood features before \emph{max} aggregation and the additional computation of subtraction is negligible.}
\section{Experiments on 3D Point Clouds}\label{sec:experiments}
We propose \emph{ResGCN} and \emph{DenseGCN} to handle the vanishing gradient problem of GCNs. To enlarge the receptive field, we define a dilated graph convolution operator for GCNs. To evaluate our framework, we conduct extensive experiments on the tasks of semantic segmentation and part segmentation on large-scale 3D point cloud datasets and demonstrate that our methods significantly improve performance. In addition, we perform a comprehensive ablation study to show the effect of different components of our framework.
\subsection{Graph Learning on 3D Point Clouds}
\label{sec:PCGCN}
Point cloud segmentation is a challenging task because of the unordered and irregular structure of 3D point clouds. Normally, each point in a point cloud is represented by its 3D spatial coordinates and possibly auxiliary features such as color and/or surface normal. We treat each point as a vertex $v$ in a directed graph $\mathcal{G}$ and we use $k$-NN to construct the directed dynamic edges between points at every GCN layer (refer to \secLabel \ref{dyna}). In the first layer, we construct the input graph $\mathcal{G}_{0}$ by applying a dilated $k$-NN search to find the nearest neighbors in 3D coordinate space. At subsequent layers, we dynamically build the edges using dilated $k$-NN in feature space. For the segmentation task, we predict the categories of all the vertices at the output layer.
\subsection{Experimental Setup}
We use the overall accuracy (OA) and mean intersection over union (mIoU) across all classes as evaluation metrics. For each class, the IoU is computed as $\frac{TP}{TP+T-P}$, where $TP$ is the number of true positive points, $T$ is the number of ground truth points of that class, and $P$ is the number of predicted positive points. We perform the majority of our experiments on semantic segmentation of point clouds on the S3DIS dataset. To motivate the use of DeepGCNs, we do a thorough ablation study on area 5 of this dataset to analyze each component and provide insights. We then evaluate our proposed reference model \emph{ResGCN-28} (backbone of 28 layers with residual graph connections and stochastic dilated graph convolutions) on all 6 areas and compare it to the shallow DGCNN baseline \cite{wang2018dynamic} and other state-of-the-art methods. In order to validate that our method is general and does not depend on a specific dataset, we also show results on PartNet for the task of part segmentation of point clouds.
\subsection{Network Architectures}
As shown in \figLabel \ref{fig:pipeline}, all the network architectures in our experiments have three blocks: a GCN backbone block, a fusion block and an MLP prediction block. The GCN backbone block is the only part that differs between experiments. For example, the only difference between \emph{PlainGCN} and \emph{ResGCN} is the use of residual skip connections for all GCN layers in \emph{ResGCN}. Both have the same number of parameters. We linearly increase the dilation rate $d$ of dilated $k$-NN with network depth. For fair comparison, we keep the fusion and MLP prediction blocks the same for all architectures. %In the S3DIS semantic segmentation task,
%\MM{Were there any changes for PartNet?} \AA{No}.
The GCN backbone block takes as input a point cloud with 4096 points, extracts features by applying consecutive GCN layers to aggregate local information, and outputs a learned graph representation with 4096 vertices. The fusion and MLP prediction blocks follow a similar architecture as PointNet \cite{pc_qi2017pointnet} and DGCNN \cite{wang2018dynamic}. The fusion block is used to fuse the global and multi-scale local features. It takes as input the extracted vertex features from the GCN backbone block at every GCN layer and concatenates those features, then passes them through a 1$\times$1 convolution layer followed by max pooling. The latter layer aggregates the vertex features of the whole graph into a single global feature vector, which in return is concatenated with the feature of each vertex from all previous GCN layers (fusion of global and local information). The MLP prediction block applies three MLP layers to the fused features of each vertex/point to predict its category. In practice, these layers are 1$\times$1 convolutions.
\mysection{PlainGCN} This baseline model consists of a \emph{PlainGCN} backbone block, a fusion block, and an MLP prediction block.
The backbone stacks EdgeConv \cite{wang2018dynamic} layers with dynamic $k$-NN, each of which is similar to the one used in DGCNN \cite{wang2018dynamic}.
No skip connections are used here.
\mysection{ResGCN} We construct \emph{ResGCN} by adding dynamic dilated $k$-NN and residual graph connections to \emph{PlainGCN}. The connections between all GCN layers in the GCN backbone block do not increase the number of parameters.
\mysection{DenseGCN} Similarly, \emph{DenseGCN} is built by adding dynamic dilated $k$-NN and dense graph connections to the \emph{PlainGCN}. As described in \secLabel \ref{sec:DenseGCN}, dense graph connections are created by concatenating all the intermediate graph representations from previous layers. The dilation rate schedule of our \emph{DenseGCN} is the same as for \emph{ResGCN}.
\subsection{Implementation}
For semantic segmentation on S3DIS \cite{2017arXiv170201105A}, we implement our models using TensorFlow, and for part segmentation on PartNet \cite{Mo_2019_CVPR}, we implement them using PyTorch. For fair comparison, we use the Adam optimizer with the same initial learning rate $0.001$ and the same learning rate schedule for all experiments; the learning rate decays $50\%$ every $3\times 10^5$ gradient decent steps. Batch normalization is applied to every layer. Dropout with a rate of $0.3$ is used at the second MLP layer of the MLP prediction block. As mentioned in \secLabel \ref{sec:dilation}, we use dilated $k$-NN with a random uniform sampling probability $\epsilon=0.2$ for GCNs with dilations. In order to isolate the effect of the proposed DeepGCN architectures, we do not use any data augmentation or post processing techniques. We train our models end-to-end from scratch for $100$ epochs. We evaluate every $10^{\text{th}}$ epoch on the test set and report the best result for each model. The networks are trained with two NVIDIA Tesla V100 GPUs using data parallelism in semantic segmentation on S3DIS, and the batch size is set to $8$ for each GPU. For part segmentation on PartNet, we set the batch size to $7$ and the networks are trained with one NVIDIA Tesla V100 GPU.
\begin{table*}[!htb]
\centering
\footnotesize %\small
\setlength{\tabcolsep}{5pt} % 5pt
\begin{tabular}{c|lcc|cccccccc}
\toprule
\textbf{\rot{Ablation}} & \textbf{\rot{Model}} & \textbf{\rot{Operator}} & \textbf{\rot{mIoU}} & \textbf{\rot{$\Delta$mIoU}} & \textbf{\rot{dynamic}} & \textbf{\rot{connection}} & \textbf{\rot{dilation}} & \R{\textbf{\rot{sto. eps.}}} & \textbf{\rot{\# NNs}} & \textbf{\rot{\# filters}} & \textbf{\rot{\# layers}} \\
\midrule
\textbf{Reference} & \emph{ResGCN-28} & EdgeConv & \textbf{52.49} & 0.00 & \checkmark & \res & \checkmark & 0.2 & 16 & 64 & 28 \\
\midrule
\multirow{7}{*}{\textbf{Dilation}} & & EdgeConv & 51.98 & -0.51 & \checkmark & \res & \checkmark & 0.0 & 16 & 64 & 28 \\
& & \R{EdgeConv} & \R{52.98} & \R{0.49} & \checkmark & \res & \checkmark & \R{0.4} & 16 & 64 & 28 \\
& & \R{EdgeConv} & \R{51.92} & \R{-0.57} & \checkmark & \res & \checkmark & \R{0.6} & 16 & 64 & 28 \\
& & \R{EdgeConv} & \R{52.35} & \R{-0.14} & \checkmark & \res & \checkmark & \R{0.8} & 16 & 64 & 28 \\
& & \R{EdgeConv} & \R{51.40} & \R{-1.08} & \checkmark & \res & \checkmark & \R{1.0} & 16 & 64 & 28 \\
& & EdgeConv & 49.64 & -2.85 & \checkmark & \res & & & 16 & 64 & 28 \\
& \emph{PlainGCN-28} & EdgeConv & \textbf{40.31} & -12.18 & \checkmark & \nc & & & 16 & 64 & 28 \\
\midrule
\multirow{2}{*}{\textbf{Fixed $k$-NN}} & & EdgeConv & 48.38 & -4.11 & & \res & & & 16 & 64 & 28 \\
& & EdgeConv & 43.43 & -9.06 & & \nc & & & 16 & 64 & 28 \\
\midrule
\multirow{6}{*}{\textbf{Connections}} & \emph{DenseGCN-28} & EdgeConv & \textbf{51.27} & -1.22 & \checkmark & \dense & \checkmark & 0.2 & 8 & 32 & 28 \\
& \emph{\R{ResGCN-$28^\dagger$}} & \R{EdgeConv} & \R{50.86} & \R{-1.63} & \checkmark & \res & \checkmark & 0.2 & 16 & 64 & 28 \\
& & EdgeConv & 40.47 & -12.02 & \checkmark & \nc & \checkmark & 0.2 & 16 & 64 & 28 \\
& & EdgeConv & 38.79 & -13.70 & \checkmark & \nc & \checkmark & 0.2 & 8 & 64 & 56 \\
& & EdgeConv & 49.23 & -3.26 & \checkmark & \nc & \checkmark & 0.2 & 16 & 64 & 14 \\
& & EdgeConv & 47.92 & -4.57 & \checkmark & \nc & \checkmark & 0.2 & 16 & 64 & 7 \\
\midrule
\multirow{3}{*}{\textbf{Neighbors}}& & EdgeConv & 49.98 & -2.51 & \checkmark & \res & \checkmark & 0.2 & 8 & 64 & 28 \\
& & EdgeConv & 49.22 & -3.27 & \checkmark & \res & \checkmark & 0.2 & 4 & 64 & 28 \\
& & EdgeConv & 49.18 & -3.31 & \checkmark & \res & \checkmark & 0.2 & 32 & 32 & 28 \\
\midrule
\multirow{4}{*}{\textbf{Depth}} %& \emph{PlainGCN-112} & EdgeConv & \textbf{29.61} & -22.88 & \checkmark & \nc & & & 4 & 64 & 112 \\
& \emph{ResGCN-112} & EdgeConv & 51.97 & -0.52 & \checkmark & \res & \checkmark & 0.2 & 4 & 64 & 112 \\
& \emph{ResGCN-56} & EdgeConv & \textbf{53.64} & 1.15 & \checkmark & \res & \checkmark & 0.2 & 8 & 64 & 56 \\
& \emph{ResGCN-14} & EdgeConv & 49.90 & -2.59 & \checkmark & \res & \checkmark & 0.2 & 16 & 64 & 14 \\
& \emph{ResGCN-7} & EdgeConv & 48.95 & -3.53 & \checkmark & \res & \checkmark & 0.2 & 16 & 64 & 7 \\
\midrule
\multirow{3}{*}{\textbf{Width}} & \emph{ResGCN-28W} & EdgeConv & \textbf{53.78} & 1.29 & \checkmark & \res & \checkmark & 0.2 & 8 & 128 & 28 \\
& & EdgeConv & 48.80 & -3.69 & \checkmark & \res & \checkmark & 0.2 & 16 & 32 & 28 \\
& & EdgeConv & 45.62 & -6.87 & \checkmark & \res & \checkmark & 0.2 & 16 & 16 & 28 \\
\midrule
\multirow{4}{*}{\textbf{GCN Variants}} & & GraphSAGE & {49.20} & -3.29 & \checkmark & \res & \checkmark & 0.2 & 16 & 64 & 28 \\
& & GraphSAGE-N & {49.02} & -3.47 & \checkmark & \res & \checkmark & 0.2 & 16 & 64 & 28 \\
& & GIN-$\epsilon$ & {42.81} & -9.68 & \checkmark & \res & \checkmark & 0.2 & 16 & 64 & 28 \\
& \emph{ResMRGCN-28} & MRGCN & \textbf{51.17} & -1.32 & \checkmark & \res & \checkmark & 0.2 & 16 & 64 & 28 \\
\bottomrule
\end{tabular}
\vspace{3pt}
\caption{\textbf{Ablation study on area 5 of S3DIS}. We compare our reference network (\emph{ResGCN-28}) with 28 layers, residual graph connections, and dilated graph convolutions to several ablated variants. All models were trained with the same hyper-parameters for 100 epochs on all areas except for area 5, which is used for evaluation. We denote residual and dense connections with the \res~ and \dense~ symbols respectively. \R{$\dagger$ denotes that residual connections are added between every two GCN layers}. We highlight the most important results in bold. $\Delta$mIoU denotes the difference in mIoU with respect to the reference model \emph{ResGCN-28}.}
\label{tbl:ablation_area5}
\end{table*}
\subsection{Results}
For convenient referencing, we use the naming convention \emph{BackboneBlock-\#Layers} to denote the key models in our analysis. We focus on residual graph connections for our analysis, since \emph{ResGCN-28} is easier and faster to train, but we expect that our observations also hold for dense graph connections.
\subsubsection{Semantic Segmentation on S3DIS}
In order to thoroughly evaluate the ideas proposed in this paper, we conduct extensive experiments on the Stanford large-scale 3D Indoor Spaces Dataset (S3DIS), a large-scale indoor dataset for 3D semantic segmentation of point clouds. S3DIS covers an area of more than $6,000 m^2$ with semantically annotated 3D meshes and point clouds. In particular, the dataset contains 6 large-scale indoor areas represented as colored 3D point clouds with a total of 695,878,620 points. As is common practice, we train networks on 5 out of the 6 areas and evaluate them on the left out area.
We begin with the extensive ablation study, where we evaluate the trained models on area 5, after training them on the other areas. Our aim is to shed light on the contribution of each component of our novel network architectures. To this end, we investigate the performance of different \textit{ResGCN} architectures, \eg with dynamic dilated $k$-NN, with regular dynamic $k$-NN (without dilation), and with fixed edges. We also study the effect of different parameters, \eg number of $k$-NN neighbors (4, 8, 16, 32), number of filters (32, 64, 128), and number of layers (7, 14, 28, 56). To ensure that our contributions (residual/dense connections and dilated graph convolutions) are general, we apply them to multiple GCN variants that have been proposed in the literature. Overall, we conduct 25 experiments and report detailed results in \tblLabel \ref{tbl:ablation_area5}. We also summarize the most important insights of the ablation study in \figLabel \ref{fig:ablation}. In the following, we discuss each block of experiments.
\begin{figure} [!htb]
\centering
\includegraphics[trim = 2mm 0mm 0mm 0mm, clip, width=\columnwidth]{figures/AblationStudyArea5_Serif.pdf}
\caption{\textbf{Ablation study on area 5 of S3DIS}. We compare our reference network (\emph{ResGCN-28}) with 28 layers, residual graph connections, and dilated graph convolutions with several ablated variants. All models were trained with the same hyper-parameters for 100 epochs on all areas except for area 5 of the S3DIS dataset.}
\label{fig:ablation}
\end{figure}
\mysection{Effect of residual graph connections}
Our experiments in \tblLabel \ref{tbl:ablation_area5} (\textit{Reference})
show that residual graph connections play an essential role in training deeper networks, as they tend to result in more stable gradients. This is analogous to the insight from CNNs \cite{he2016deep}. \R{ResNet \cite{he2016deep} adds skip connections between every two convolutional blocks; we conduct experiments with two skip connection variants, \emph{ResGCN-28} and \emph{ResGCN-$28^\dagger$}. \emph{ResGCN-28} adds skip connections between every GCN layer while \emph{ResGCN-$28^\dagger$} adds skip connections between every two GCN layers. We find that \emph{ResGCN-28} outperforms \emph{ResGCN-$28^\dagger$} by $1.63\%$ mIOU (refer to \tblLabel \ref{tbl:ablation_area5}). Thus, we adopt skip connections between every layer for all other ResGCN architectures.} When the residual graph connections between layers are removed (\ie in \emph{PlainGCN-28}), performance dramatically degrades (-12\% mIoU). \figLabel \ref{fig:plainVSres} shows the importance of residual graph connections very clearly. As network depth increases, skip connections become critical for convergence. We also show similar performance gains by combining residual graph connections and dilated graph convolutions with other types of GCN layers. These results can be seen in the ablation study \tblLabel \ref{tbl:ablation_area5} (\textit{GCN Variants}) and are further discussed later in this section.
\begin{figure} [!htb]
\centering
\includegraphics[trim = 5mm 0mm 5mm 0mm, clip, width=\columnwidth]{figures/plainVSres_bar_labels.pdf}\vspace{-12pt}
\caption{\textbf{PlainGCN vs. ResGCN on area 5 of S3DIS}. We compare networks of different depths with and without \emph{residual graph connections}. All models were trained for 100 epochs on all areas except for area 5 with the same hyper-parameters. Only when residual graph connections are used do the results improve with increasing depth.}
\label{fig:plainVSres}
\end{figure}
\mysection{Effect of dilation}
Results in \tblLabel \ref{tbl:ablation_area5} (\textit{Dilation}) \cite{yu2015multi} show that dilated graph convolutions account for a 2.85\% improvement in mean IoU (\textit{row 3}),
justified primarily by the expansion of the network's receptive field.
We find that adding stochasticity to the dilated $k$-NN does helps performance but not to a significant extent. \R{We also investigate the effect of $\epsilon$ for \emph{stochastic dilation} and report the results in \tblLabel \ref{tbl:ablation_area5} (\textit{sto. eps.}). We find that models are robust to different values of $\epsilon$. The mIOUs vary in the range of about $1\%$ for different $\epsilon$ values. It is possible to achieve better performance by carefully tuning $\epsilon$. For instance, \emph{ResGCN-28} with $\epsilon$ set to $0.4$ increases the mIOU by 0.49\% compared the \textit{Reference} model with $\epsilon$ equal to $0.2$.}
Interestingly, our results in \tblLabel \ref{tbl:ablation_area5}
also indicate that dilation especially helps deep networks when combined with residual graph connections (\textit{rows 1,13}). Without such connections, performance can actually degrade with dilated graph convolutions. The reason for this is probably that these varying neighbors result in `worse' gradients, which further hinder convergence when residual graph connections are not used.
\mysection{Effect of dynamic $k$-NN}
While we observe an improvement when updating the $k$ nearest neighbors after every layer, we would also like to point out that it comes at a relatively high computational cost. We show different variants without dynamic edges in \tblLabel \ref{tbl:ablation_area5} (\textit{Fixed $k$-NN}).
\mysection{Effect of dense graph connections}
We observe similar performance gains with dense graph connections (\emph{DenseGCN-28}) in \tblLabel \ref{tbl:ablation_area5} (\textit{Connections}). However, with a naive implementation, the memory cost is prohibitive. Hence, the largest model we can fit into GPU memory uses only $32$ filters and $8$ nearest neighbors, as compared to $64$ filters and $16$ neighbors in the case of its residual counterpart \emph{ResGCN-28}. Since the performance of these two deep GCN variants is similar, residual connections are more practical for most use cases and hence we focus on them in our ablation study. Yet, we do expect the same insights to transfer to the case of dense graph connections.
\mysection{Effect of nearest neighbors} Results in \tblLabel \ref{tbl:ablation_area5} (\textit{Neighbors}) show that a larger number of neighbors helps in general. As the number of neighbors is decreased by a factor of 2 and 4, the performance drops by 2.5\% and 3.3\% respectively. However, a large number of neighbors only results in a performance boost, if the network capacity is sufficiently large. This becomes apparent when we increase the number of neighbors by a factor of 2 and decrease the number of filters by a factor of 2 (refer to \textit{row 3} in \textit{Neighbors}).
\mysection{Effect of network depth}
\tblLabel \ref{tbl:ablation_area5} (\textit{Depth}) shows that increasing the number of layers improves network performance, but only if residual graph connections and dilated graph convolutions are used, as is clearly shown in \tblLabel \ref{tbl:ablation_area5} (\textit{Connections}).
\mysection{Effect of network width} Results in \tblLabel \ref{tbl:ablation_area5} (\textit{Width}) show that increasing the number of filters leads to a similar increase in performance as increasing the number of layers. In general, a higher network capacity enables learning nuances necessary for succeeding in corner cases.
\mysection{GCN Variants}
Our experiments in \tblLabel \ref{tbl:ablation_area5} (\textit{GCN Variants}) show the effect of using different GCN operators. The results clearly show that different deep GCN variants with residual graph connections and dilated graph convolutions converge better than \emph{PlainGCN}. Using our proposed \emph{MRGCN} operator achieves comparable performance to the \emph{ResGCN} reference model, which relies on \emph{EdgeConv}, while only using half the GPU memory. The \emph{GraphSAGE} operator performs slightly worse and our results also show that using normalization (\ie \emph{GraphSAGE-N}) is not essential. Interestingly, when using the \emph{GIN-$\epsilon$} operator, the network converges well during the training phase and has a high training accuracy but fails to generalize to the test set. This phenomenon is also observed in the original paper \cite{xu2018powerful}, in which the best performance is achieved when $\epsilon$ is set to $0$.
\mysection{Qualitative Results}
\figLabel \ref{fig:qualitative} shows qualitative results on area 5 of S3DIS \cite{2017arXiv170201105A}. As expected from the results in \tblLabel \ref{tbl:ablation_area5}, our \emph{ResGCN-28} and \emph{DenseGCN-28} perform particularly well on difficult classes such as board, beam, bookcase and door. \emph{Rows 1-4} clearly show how \emph{ResGCN-28} and \emph{DenseGCN-28} are able to segment the board, beam, bookcase and door respectively, while \emph{PlainGCN-28} completely fails. Please refer to the \supp for more qualitative results and further results.
\begin{figure*}[!h]
\centering
\includegraphics[page=1, width=\textwidth]{figures/gcn_qualitative_res_dense_camera_ready_with_28.pdf}
\caption{\textbf{Qualitative Results on S3DIS Semantic Segmentation}. We show here the effect of adding residual and dense graph connections to deep GCNs. \emph{PlainGCN-28}, \emph{ResGCN-28}, and \emph{DenseGCN-28} are identical except for the presence of residual graph connections in \emph{ResGCN-28} and dense graph connections in \emph{DenseGCN-28}. We note how both residual and dense graph connections have a substantial effect on hard classes like board, bookcase, and sofa. These are lost in the results of \emph{PlainGCN-28}.}
\label{fig:qualitative}
\end{figure*}
\begin{figure*}[!h]
\centering
\includegraphics[page=1,trim = 0mm 0mm 200mm 300mm, clip, width=3.55in]{figures/Plain28Res28.pdf}
\includegraphics[page=3,trim = 0mm 0mm 200mm 300mm, clip, width=3.55in]{figures/Plain28Res28.pdf}
\includegraphics[page=2,trim = 0mm 0mm 200mm 600mm, clip, width=3.55in]{figures/Plain28Res28.pdf}
\includegraphics[page=4,trim = 0mm 0mm 200mm 600mm, clip, width=3.55in]{figures/Plain28Res28.pdf}
\caption{\textbf{Qualitative Results on PartNet Part Segmentation}. We show here the effect of adding residual connections to deep GCNs. \emph{PlainGCN-28} and \emph{ResGCN-28} are identical except for the presence of residual connections in \emph{ResGCN-28}. We note how residual connections have a positive effect on part segmentation compared to \emph{PlainGCN-28}. Many important parts of the objects are classified incorrectly using \emph{PlainGCN-28}. }
\label{fig:qualitative_partnet}
\end{figure*}
\begin{table*}[!htb]
\centering
\footnotesize % \small
\setlength{\tabcolsep}{4.2pt} % 3pt
\begin{tabular}{lcc|ccccccccccccc}
\toprule
\textbf{Method} & \textbf{OA} & \textbf{mIOU} & \textbf{ceiling} & \textbf{floor} & \textbf{wall} & \textbf{beam} & \textbf{column} & \textbf{window} & \textbf{door} & \textbf{table} & \textbf{chair} & \textbf{sofa} & \textbf{bookcase} & \textbf{board} & \textbf{clutter} \\
\midrule
PointNet \cite{pc_qi2017pointnet}& 78.5 & 47.6 & 88.0 & 88.7 & 69.3 & 42.4 & 23.1 & 47.5 & 51.6 & 54.1 & 42.0 & 9.6 & 38.2 & 29.4 & 35.2 \\
MS+CU \cite{3dsemseg_ICCVW17} & 79.2 & 47.8 & 88.6 & \textbf{95.8} & 67.3 & 36.9 & 24.9 & 48.6 & 52.3 & 51.9 & 45.1 & 10.6 & 36.8 & 24.7 & 37.5 \\
G+RCU \cite{3dsemseg_ICCVW17} & 81.1 & 49.7 & 90.3 & 92.1 & 67.9 & \textbf{44.7} & 24.2 & 52.3 & 51.2 & 58.1 & 47.4 & 6.9 & 39.0 & 30.0 & 41.9 \\
PointNet++ \cite {pc_qi2017pointnet++} &- &53.2 &90.2 &91.7 &73.1 &42.7 &21.2 &49.7 &42.3 &62.7 &59.0 &19.6 &45.8 &48.2 &45.6\\
%SPG \cite{SuperpointCVPR18} & 85.5 & 62.1 & 89.9 & 95.1 & 76.4 & 62.8 & 47.1 & 55.3 & 68.4 & 69.2 & 73.5 & 45.9 & 63.2 & 8.7 & 52.9 \\
3DRNN+CF \cite{pc_ye20183d}& \textbf{86.9} & 56.3 & 92.9 & 93.8 & 73.1 & 42.5 & 25.9 & 47.6 & 59.2 & 60.4 & \textbf{66.7} & 24.8 & \textbf{57.0} & 36.7 & 51.6 \\
\midrule
DGCNN \cite{wang2018dynamic} & 84.1 & 56.1 & - & - & - & - & - & - & - & - & - & - & - & - & - \\
\textbf{ResGCN-28 (\textit{Ours})} & 85.9 & \textbf{60.0} & \textbf{93.1} & 95.3 & \textbf{78.2} & 33.9 & \textbf{37.4} & \textbf{56.1} & \textbf{68.2} & \textbf{64.9} & 61.0 & \textbf{34.6} & 51.5 & \textbf{51.1} & \textbf{54.4}\\
\bottomrule
\end{tabular}
\vspace{3pt}
\caption{\textbf{Comparison of \emph{ResGCN-28} with the state-of-the-art on S3DIS Semantic Segmentation}. We report average per-class results across all areas for our reference model \emph{ResGCN-28} and state-of-the-art baselines. \emph{ResGCN-28} which has 28 GCN layers, residual graph connections, and dilated graph convolutions outperforms the previous state-of-the-art by almost $4\%$. It also outperforms all baselines in $9$ out of $13$ classes. The metrics shown are overall point accuracy (OA) and mean IoU (mIoU). '-' denotes not reported and \textbf{bold} denotes best performance.}
\label{tbl:ours_vs_baselines}
\end{table*}
\mysection{Comparison to state-of-the-art}
Finally, we compare our reference network (\emph{ResGCN-28}), which incorporates the ideas put forward in the methodology, to several state-of-the-art baselines in \tblLabel \ref{tbl:ours_vs_baselines}. The results clearly show the effectiveness of deeper models with residual graph connections and dilated graph convolutions. \emph{ResGCN-28} outperforms DGCNN \cite{wang2018dynamic} by 3.9\% (absolute) in mean IoU. DGCNN has the same fusion and MLP prediction blocks as \emph{ResGCN-28} but a shallower \emph{PlainGCN}-like backbone block.
Furthermore, we outperform all baselines in 9 out of 13 classes. We perform particularly well in the difficult object classes such as board, where we achieve 51.1\%, and sofa, where we improve state-of-the-art by about 10\% mIOU.
This significant performance improvement on the difficult classes is probably due to the increased network capacity, which allows the network to learn subtle details necessary to distinguish between a board and a wall for example. The first row in \figLabel \ref{fig:qualitative} is a representative example for this occurrence.
Our performance gains are solely due to our innovation in the network architecture, since we use the same hyper-parameters and even learning rate schedule as the baseline DGCNN \cite{wang2018dynamic} and only decrease the number of nearest neighbors from $20$ to $16$ and the batch size from $24$ to $16$ due to memory constraints. We outperform state-of-the art methods by a significant margin and expect further improvement from tweaking the hyper-parameters, especially the learning schedule.
\begin{table*}[!htb]
\centering
\footnotesize % \small
\setlength{\tabcolsep}{2.7pt} % 3pt
\begin{tabular}{l|c|cccccccccccccccccccc}
\toprule
\textbf{Method} & \R{\textbf{Avg}} & \textbf{bed} & \textbf{bottle} & \textbf{chair} & \textbf{clock} & \textbf{dishw.} & \textbf{disp.} & \textbf{door} & \textbf{earph.} & \textbf{fauc.} & \textbf{knife} & \textbf{lamp} & \textbf{micro.} & \textbf{fridge} & \textbf{st. furn.} & \textbf{table} & \textbf{tr. can} & \textbf{vase} \\
\midrule
PointNet \cite{pc_qi2017pointnet} & 35.6& 13.4 & 29.5 & 27.8 & 28.4 & 48.9 & 76.5 & 30.4 & 33.4 & 47.6 & 32.9 & 18.9 & 37.2 & 33.5 & 38.0 & 29.0 & 34.8 & 44.4 \\
PointNet++ \cite{pc_qi2017pointnet++} & 42.5 & 30.3 & 41.4 & 39.2 & {41.6} & 50.1 & 80.7 & 32.6 & 38.4 & 52.4& 34.1 & {25.3} & 48.5 & 36.4 & 40.5 & {33.9} & 46.7 & 49.8 \\
SpiderCNN \cite{xu2018spidercnn}& 37.0 & 36.2 & 32.2 & 30.0 & 24.8 & 50.0 & 80.1 & 30.5 & 37.2 & 44.1& 22.2 & 19.6 & 43.9 & 39.1 & 44.6 & 20.1 & 42.4 & 32.4 \\
PointCNN \cite{Li2018PointCNNCO} & 46.5& {41.9}& 41.8& {43.9}& 36.3& 58.7& {82.5}& {37.8}& {48.9} & {60.5}& 34.1& 20.1& {58.2}& 42.9& {49.4}& 21.3& {53.1}& 58.9 \\
\midrule
\R{\textbf{PlainGCN-28 (\textit{Ours})}} & 31.3 & 20.1& 29.4& 24.6& 21.4& 38.2& 73.2& 24.2& 36.4& 42.2& 32.5& 16.7& 35.6& 29.8& 30.1& 15.1& 33.8& 28.2\\
\R{\textbf{ResGCN-28 (\textit{Ours})}} &45.1 & 35.9 & 49.3 & 41.1 &33.8 &56.2 &81.0 &31.1 &45.8 &52.8 &44.5 &23.1 &51.8 &34.9 &47.2 &33.6 &50.8 &54.2 \\
\bottomrule
\end{tabular}
\vspace{3pt}
\caption{\textbf{Comparison of \emph{ResGCN-28} with other methods on PartNet Part Segmentation.} We report the part-category mean IoU (mIoU) on the fine-grained level of segmentation. \emph{ResGCN-28} which has 28 GCN layers, residual graph connections, and dilated graph convolutions outperforms \emph{PlainGCN-28} and most previous methods w.r.t the average mIoU.
}
\label{tbl:partnet_results}
\end{table*}
\subsubsection{Part Segmentation on PartNet}
We further experiment with our architecture on the task of part segmentation and evaluate it on the recently proposed large-scale PartNet \cite{Mo_2019_CVPR} dataset. PartNet consists of over 26,671 3D models from 24 object categories with 573,585 annotated part instances. The dataset establishes three benchmarking tasks for part segmentation on 3D objects: fine-grained semantic segmentation, hierarchical semantic segmentation, and instance segmentation. For the following experiments, we focus on the fine-grained level of semantic segmentation, which includes 17 out of the 24 object categories present in the PartNet dataset.
\R{We use the same reference architecture, \emph{ResGCN-28}, that we used for the experiments on the S3IDS dataset. We compare its performance to the baseline architecture \emph{PlainGCN-28} to show the impact of our residual connections and stochastic dilated convolutions. We also compare against the state-of-the-art reported in the PartNet paper \cite{Mo_2019_CVPR}, namely PointNet \cite{pc_qi2017pointnet}, PointNet++ \cite{pc_qi2017pointnet++}, SpiderCNN \cite{xu2018spidercnn}, and PointCNN \cite{Li2018PointCNNCO}. As suggested in PartNet \cite{Mo_2019_CVPR}, we use 10,000 sampled points as input. We train a separate network for each category and optimize them with Adam for $500$ epochs without weight decay. The initial learning rate is $0.005$ that is decayed by a factor of $0.9$ every $50$ epochs. We report our results using the best pretrained models on the validation sets. }
\mysection{Qualitative results} \figLabel \ref{fig:qualitative_partnet} shows qualitative results on 4 categories of PartNet \cite{Mo_2019_CVPR}: bottle, bed, microwave, and refrigerator. As expected from the results in Table \ref{tbl:partnet_results}, \emph{ResGCN-28} performs very well compared to the baseline \emph{PlainGCN-28}, where there are no residual connections between layers. Although \emph{ResGCN-28} produces some incorrect outputs compared to the ground truth in categories like microwave and bed, it still outperforms \emph{PlainGCN-28} and segments the important parts of the object.
We provide more qualitative results in the \supp.
\mysection{Comparison to state-of-the-art}
\R{
We summarize the results of our \emph{ResGCN-28} and compare it to \emph{PlainGCN-28} and other state-of-the-art methods in \tblLabel \ref{tbl:partnet_results}. \emph{ResGCN-28} outperforms \emph{PlainGCN-28} by a large margin, which illustrates how our proposed residual connections and stochastic dilated convolutions enable training deep GCN architectures.
\emph{ResGCN-28} also substantially outperforms PointNet \cite{pc_qi2017pointnet}, PointNet++ \cite{pc_qi2017pointnet++}, and SpiderCNN \cite{xu2018spidercnn}. Note that PointCNN designs a point convolution specialized for point clouds, whereas \emph{ResGCN-28} is a general framework for any graph (\eg point clouds, biological networks, citation networks, \etc). Despite this huge disadvantage, \emph{ResGCN-28} still achieves results on par with PointCNN \cite{Li2018PointCNNCO} and even outperforms PointCNN on some categories including bottle, knife, lamp, and table.
}
\R{\subsubsection{Object Classification on ModelNet40}}
\R{We experiment with our architecture on the task of object classification and evaluate it on the popular object classification dataset ModelNet40 \cite{ben20183dmfv}. The dataset contains 12,311 CAD models from 40 different categories. Although saturated, ModelNet40 is considered an important benchmark for the task of 3D object classification. The results in \tblLabel \ref{tb: modelnet40} show the superiority of our \emph{ResGCN} and \emph{DenseGCN} architectures over the baseline \emph{PlainGCN} and state-of-the-art methods. When compared to the baseline \emph{PlainGCN}, one observes the positive effect of our residual connections, dense connections, and dilated convolutions. In addition, the \emph{ResGCN} and \emph{DenseGCN} architectures outperform all state-of-the-art approaches in overall test accuracy. Particularly, \emph{ResGCN-14} achieves 93.6\% on ModelNet-40 in terms of overall accuracy. Here, we point out that the recent work of KPConv \cite{Thomas2019KPConvFA} uses a much larger number of trainable parameters compared to our reference \emph{ResGCN-28}, as shown in \tblLabel \ref{tb: modelnet40}. And yet, even our shallower variant \emph{ResGCN-14} can outperform KPConv, while using almost one order of magnitude less trainable parameters.}
\vspace{9pt}
% sota comparasion
\begin{table}[!htb]
\centering
\setlength{\tabcolsep}{5pt} % 3pt
\begin{tabular}{l|ccc}
\toprule
\R{\textbf{Model}} & \R{\textbf{O.A. (\%)}} & \R{\textbf{C.A. (\%)}} & \R{\textbf{Param (M)}} \\
\midrule
\R{3DmFV-Net \cite{ben20183dmfv}} & \R{91.6} & \R{-} & \R{45.77}\\
\R{PointNet \cite{pc_qi2017pointnet}} & \R{89.2} & \R{86.2} & \R{3.5}\\
\R{PointNet++ \cite{pc_qi2017pointnet++}} & \R{90.7} & \R{-} & \R{1.48}\\
\R{SO-Net \cite{Li2018SONetSN}} & \R{90.9} & \R{87.3} & \R{-}\\
\R{PCNN by Ext \cite{Atzmon2018PointCN}} & \R{92.3} & \R{-} & \R{8.2}\\
\R{SpecGCN \cite{Wang2018LocalSG}} & \R{91.5} & \R{-} & \R{2.05}\\
\R{SpiderCNN \cite{xu2018spidercnn}} & \R{90.5} & \R{-} & \R{-}\\
\R{MCConv \cite{Hermosilla2018MonteCC}} & \R{90.9} & \R{-} & \R{-}\\
\R{FlexConv \cite{Groh2018FlexConvolutionM}} & \R{90.2} & \R{-} & \R{0.35}\\
\R{PointCNN \cite{Li2018PointCNNCO}} & \R{92.2} & \R{88.1} & \R{0.6}\\
\R{DGCNN \cite{wang2018dynamic}} & \R{92.9} & \R{90.2} & \R{1.84}\\
\R{KPConv rigid \cite{Thomas2019KPConvFA}} & \R{92.9} & \R{-} & \R{14.3}\\
\R{KPConv deform \cite{Thomas2019KPConvFA}} & \R{92.7} & \R{-} & \R{15.2}\\
\midrule
\R{\textbf{\emph{PlainGCN-14} (\textit{Ours})}} & \R{90.5} & \R{85.1}& \R{2.2}\\
\R{\textbf{\emph{PlainGCN-28} (\textit{Ours})}} & \R{91.4} & \R{86.8}& \R{3.3}\\
\R{\textbf{\emph{ResGCN-14} (\textit{Ours})}} & \textbf{\R{93.6}} & \textbf{\R{90.9}}& \R{2.2}\\
\R{\textbf{\emph{ResGCN-28} (\textit{Ours})}} & \R{\textbf{93.3}} &\R{89.5 }& \R{3.3}\\
\R{\textbf{\emph{DenseGCN-14} (\textit{Ours})}} &\R{\textbf{93.3}} &\R{89.4}& \R{8.8}\\
\R{\textbf{\emph{DenseGCN-28} (\textit{Ours})}} &\R{\textbf{93.2}} &\R{\textbf{90.3}}& \R{30.9}\\
\bottomrule
\end{tabular}
\vspace{3pt}
\caption{\R{\textbf{Comparison of our GCN variants with the state-of-the-art on ModelNet40 point cloud classification}. We report the overall classification test accuracy (O.A.) and mean per-class accuracy (C.A.) of our models and state-of-the-art models in addition to the to total number of parameters for each model. One can observe the superiority of our \emph{ResGCN} and \emph{DenseGCN} architectures over state-of-the-art.}}
\label{tb: modelnet40}
\end{table}
\section{Experiments on Biological Networks}
\label{sec:experiments2}
In order to demonstrate the generality of our contributions and specifically our deep \emph{ResGCN} architecture, we conduct further experiments on general graph data. We choose the popular task of node classification on biological graph data, which is quite different from point cloud data.
In the following experiments, we mainly study the effects of skip connections, the number of GCN layers (\ie depth), number of filters per layer (\ie width) and different graph convolutional operators.
\subsection{Graph Learning on Biological Networks}
\label{sec:exp2-PPIGCN}
The main difference between biological networks and point cloud data is that biological networks have inherent edge information and high dimensional input features.
For the graph learning task on biological networks, we use the PPI \cite{chem_zitnik2017predicting} dataset to evaluate our architectures. PPI is a popular dataset for multi-label node classification, containing 24 graphs with 20 in the training set, 2 in the validation set, and 2 in the testing set.
Each graph in PPI corresponds to a different human tissue, each node in a graph represents a protein and edges represent the interaction between proteins.
Each node has positional gene sets, motif gene sets, and immunological signatures as input features (50 in total) and 121 gene ontology sets as labels. The input of the task is a graph that contains 2373 nodes on average, and the goal is to predict which labels are contained in each node.
We use essentially the same reference architecture as for point cloud segmentation described in \secLabel \ref{sec:PCGCN}, but we predict multiple labels. The number of filters of the first and last layers are changed to adapt to this task. Instead of constructing edges by means of $k$-NN, we use the edges provided by PPI directly. If we were to construct edges dynamically, there is a chance to lose the rich information provided by the initial edges.
\subsection{Experimental Setup}
Following common procedure, we use the micro-average F1 (m-F1) score as the evaluation metric.
For each graph, the F1 score is computed as follows:
\begin{equation}
\text{F1-score} = 2 \times \frac{(precision \times recall)}{(precision+recall)},
\end{equation}
where $precision = \frac{TP'}{P'}$, $recall = \frac{TP'}{T'}$, $TP'$ is the total number of true positive points of all the classes, $T'$ is the number of ground truth positive points, and $P'$ is the number of predicted positive points.
We find the best model, \ie the one with the highest accuracy (m-F1 score) on the validation set in the training phase, and then calculate the m-F1 score across all the graphs in the test set.
We show the performance and GPU memory usage of our proposed \emph{MRGCN} and compare them with other graph convolutions, \eg EdgeConv \cite{wang2018dynamic}, GATConv \cite{velivckovic2017graph}, SemiGCN \cite{kipf2016semi} and GINConv \cite{xu2018powerful}.
We conduct an extensive ablation study on the number of filters and the number of GCN layers to show their effect in the backbone network.
Our ablation study also includes experiments to show the importance of residual graph connections and dense graph connections in our DeepGCN framework. To ensure a fair comparison, all networks in our ablation study share the same architecture.
Finally, we compare our best models to several state-of-the-art methods for this task.
\subsection{Implementation}
For this biological network node classification task, we implement all our models based on PyTorch Geometric \cite{Fey/Lenssen/2019}.
We use the Adam optimizer with the same initial learning rate $0.0002$ and learning rate schedule with learning rate decay of $80\%$ every $2,000$ gradient decent steps for all the experiments.
The networks are trained with one NVIDIA Tesla V100 GPU with a batch size of $1$. Dropout with a rate of $0.2$ is used at the first and second MLP layers of the prediction block.
For fair comparison, we do not use any data augmentation or post processing techniques.
Our models are trained end-to-end from scratch.
\subsection{Results}
We study the effect of residual and dense graph connections on multi-label node classification performance. We also investigate the influence of different parameters, \eg the number of filters $(32, 64, 128, 256)$ and layers $(3, 7, 14, 28, 56, 112)$. To show the generality of our framework, we also apply the proposed residual connections to multiple GCN variants.
\mysection{Effect of graph connections}
Results in \tblLabel \ref{tb: ppi-deep} show that both residual and dense graph connections help train deeper networks. When the network is shallow, models with graph connections achieve similar performance as models without them. However, as the network grows deeper, the performance of models without graph connections drops dramatically, while the performance of models with graph connections is stable or even improves further. For example, when the number of filters is 32 and the depth is 112, the performance of \emph{ResMRGCN-112} is nearly 37.66\% higher than \emph{PlainMRGCN-112} in terms of the m-F1 score. We note that \emph{DenseMRGCN} achieves slightly better performance than \emph{ResMRGCN} with the same network depth and width.
\mysection{Effect of network depth}
Results in \tblLabel \ref{tb: ppi-deep} show that increasing the number of layers improves network performance if residual or dense graph connections are used. Although \emph{ResMRGCN} has a slight performance drop when the number of layers reaches $112$, the m-F1 score is still much higher than the corresponding \emph{PlainMRGCN}. The performance of \emph{DenseMRGCN} increases reliably as the network grows deeper; however, \emph{DenseMRGCN} consumes more memory than \emph{ResMRGCN} due to concatenations of feature maps. Due to this memory issue, we are unable to train some models and denote them with '-' in \tblLabel \ref{tb: ppi-deep}. Meanwhile, \emph{PlainMRGCN}, which has no graph connections, only enjoys a slight performance gain as the network depth increases from $3$ to $14$. For depths beyond $14$ layers, the performance drops significantly. Clearly, using graph connections improves performance, especially for deeper networks where it becomes essential.
\mysection{Effect of network width}
Results of each row in \tblLabel \ref{tb: ppi-deep} show that increasing the number of filters can consistently increase performance. A higher number of filters can also help convergence for deeper networks. However, a large number of filters is very memory consuming. Hence, we only consider networks with up to $256$ filters in our experiments.
\mysection{Effect of GCN variants}
\tblLabel \ref{tb: ppi-convs} shows the effect of using different GCN operators with different model depths. Residual graph connections and the GCN operators are the only difference when the number of layers is kept the same. The results clearly show that residual graph connections in deep networks can help different GCN operators achieve better performance than \emph{PlainGCN}. Interestingly, when the network grows deeper, the performance of \emph{PlainSemiGCN}, \emph{PlainGAT}, and \emph{PlainGIN} decreases dramatically; meanwhile, \emph{PlainEdgeConv} and \emph{PlainMRGCN} only observe a relatively small performance drop. In comparison, our proposed \emph{MRGCN} operator achieves the best performance among all models.
\mysection{Memory usage of GCN variants}
In \figLabel \ref{fig: ppi-memory}, we compare the total memory usage and performance of different GCN operators. All these models share the the same architecture except for the GCN operations. They all use residual graph connections with $56$ layers and $256$ filters. We implement all the models with PyTorch Geometric and train each using one NVIDIA Tesla V100. The GPU memory usage is measured when the memory usage is stable. Our proposed \emph{ResMRGCN} achieves the best performance, while only using around $15$\% GPU memory compared to \emph{ResEdgeConv}.
\mysection{Comparison to state-of-the-art}
Finally, we compare our \emph{DenseMRGCN-14} and \emph{ResMRGCN-28} to several state-of-the-art baselines in \tblLabel \ref{tb: ppi-sota}. Results clearly show the effectiveness of deeper models with residual and dense graph connections. \emph{DenseMRGCN-14} and \emph{ResMRGCN-28} outperform the previous state-of-the-art Cluster-GCN \cite{chiang2019cluster} by 0.07\% and 0.05\% respectively. It is worth mentioning that a total of ten models in \tblLabel \ref{tb: ppi-deep} surpass Cluster-GCN and achieve the new state-of-the-art.
\begin{table}[!htb]
\centering
\setlength{\tabcolsep}{5pt} % 3pt
\begin{tabular}{l|cccc}
\toprule
\textbf{Number of filters} & 32 & 64 & 128 & 256 \\
\midrule
\emph{PlainMRGCN-3} & 95.84 & 97.60 & 98.58 & 99.13 \\
\emph{PlainMRGCN-7} & 97.35 & 98.69 & 99.22 & \textbf{99.38}\\
\emph{PlainMRGCN-14} & 97.55 & 99.02 & 99.31 & 99.34\\
\emph{PlainMRGCN-28} & 98.09 & 99.00 & 99.02 & 99.31\\
\emph{PlainMRGCN-56} & 92.70 & 97.43 & 97.31 & 97.61\\
\emph{PlainMRGCN-112} & 60.75 & 71.97 & 89.69 & 91.50\\
\midrule
\emph{ResMRGCN-3} & 96.04 & 97.60 & 98.53 & 99.09\\
\emph{ResMRGCN-7} & 97.00 & 98.43 & 99.19 & 99.30\\
\emph{ResMRGCN-14} & 97.75 & 98.88 & 99.26 & \textbf{99.38}\\
\emph{ResMRGCN-28} & 98.50 & 99.16 & 99.29 & \textbf{99.41}\\
\emph{ResMRGCN-56} & 98.62 & 99.27 & \textbf{99.36} & \textbf{99.40}\\
\emph{ResMRGCN-112} & 98.41 & 99.34 & \textbf{99.38} & \textbf{99.39}\\
\midrule
\emph{DenseMRGCN-3} & 95.96 & 97.85 & 98.66 & 99.11\\
\emph{DenseMRGCN-7} & 97.87 & 98.47 & 99.31 & \textbf{99.36}\\
\emph{DenseMRGCN-14} & 98.93 & 99.00 & 99.01 & \textbf{99.43}\\
\emph{DenseMRGCN-28} & 99.16 & 99.29 & \textbf{99.42} & -\\
\emph{DenseMRGCN-56} & 99.22 & - & - & -\\
\bottomrule
\end{tabular}
\vspace{3pt}
\caption{\textbf{Ablation study on graph connections, network width, and network depth}. The m-F1 score is used as the evaluation metric (in \%). We find that residual and dense connections can help deep networks converge much better compared to the same model without any connections. Also, network width is positively correlated with network performance. Note that '-' denotes models that are not applicable due to memory limitations. \textbf{Bold} highlights models that outperform all state-of-the-art baselines for this multi-label PPI node classification problem.
}
\label{tb: ppi-deep}
\end{table}
% Convolutions
\begin{table}[!htb]
\centering
\setlength{\tabcolsep}{5pt} % 3pt
\begin{tabular}{l|ccccc}
\toprule
\textbf{Number of layers} & 3 & 7 & 14 & 28 & 56\\
\midrule
\emph{PlainSemiGCN} & 97.82 & 90.40 & 80.55 & 41.00 & 50.75\\
\emph{ResSemiGCN} & 97.88 & 95.05 & 93.50 & 90.60 & 90.54\\
\midrule
\emph{PlainGAT} & 98.52 & 80.92 & 56.88 & 42.40 & 48.95\\
\emph{ResGAT} & 98.63 & 97.86 & 98.99 & 99.06 & 63.25\\
\midrule
\emph{PlainGIN} & 97.86 & 57.78 & 40.79 & 35.82 & 0.26\\
\emph{ResGIN} & 97.80 & 96.44 & 98.22 & 97.44 & 97.18\\
\midrule
\emph{PlainEdgeConv} & 99.16 & 99.27 & 99.30 & 99.33 & 98.99\\
\emph{ResEdgeConv} & 99.03 & 99.19 & 99.26 & 99.30 & 99.04\\
\midrule
\emph{PlainMRGCN} & 99.13 & \textbf{99.38} & 99.34 & 99.31 & 97.61\\
\textbf{\emph{ResMRGCN}} & 99.09 & 99.30 & \textbf{99.38} & \textbf{99.41} & \textbf{99.40}\\
\bottomrule
\end{tabular}
\vspace{3pt}
\caption{\textbf{Ablation study on network depth and GCN variants.} m-F1 score is used as the evaluation metric (in \%). We set the number of filters per layer in the backbone network to $256$ and vary the number of layers. Residual graph connections can generally help different GCN operators achieve better performance than \emph{PlainGCN} when the network grows deep. \textbf{Bold} highlights models that outperform all state-of-the-art baselines for this multi-label PPI node classification problem.
}
\label{tb: ppi-convs}
\end{table}
% sota comparasion
\begin{table}[!htb]
\centering
\setlength{\tabcolsep}{5pt} % 3pt
\begin{tabular}{l|c}
\toprule
\textbf{Model} & \textbf{m-F1 score (\%)} \\
\midrule
GraphSAGE\cite{hamilton2017inductive} & 61.20 \\
GATConv \cite{velivckovic2017graph} & 97.30 \\
VR-GCN \cite{chen2017vrgcn} & 97.80 \\
GaAN \cite{zhang2018gaan} & 98.71 \\
GeniePath \cite{liu2019geniepath} & 98.50 \\
Cluster-GCN \cite{chiang2019cluster}& 99.36 \\
\midrule
\textbf{\emph{ResMRGCN-28} (\textit{Ours})} & \textbf{99.41} \\
\textbf{\emph{DenseMRGCN-14} (\textit{Ours})} & \textbf{99.43} \\
\bottomrule
\end{tabular}
\vspace{3pt}
\caption{\textbf{Comparison of \emph{DenseMRGCN-14} with state-of-the-art on PPI node classification}. We follow convention and compare models based on the m-F1 score. Our model \emph{DenseMRGCN-14}, which has 14 MRGCN layers, 256 filters in the first layer, and dense graph connections outperforms all baselines. Our \emph{ResMRGCN-28}, which has 28 MRGCN layers, 256 filters per layer, and residual graph connections also outperforms previous state-of-the-art.}
\label{tb: ppi-sota}
\end{table}
\begin{figure}[!t]
\centering
\begin{tabular}{cc}
\includegraphics[trim=5mm 0mm 0mm 0mm, width=1\columnwidth]{figures/memcompare.pdf}
\end{tabular}
\caption{\textbf{Memory usage for different GCNs on PPI node classification.} We keep the same parameters for different models and compare their m-F1 score with their total memory usage. All models in the comparison have 256 filters per layer and 56 layers. We notice that our model \emph{ResMRGCN} uses approximately 1/6 of the total memory used by \emph{ResEdgeConv} and gives a better m-F1 score.}
\label{fig: ppi-memory}
\end{figure}
\section{Conclusion}\label{sec:conclusion}
This work shows how proven concepts from CNNs (\ie residual/dense connections and dilated convolutions) can be transferred to GCNs in order to make GCNs go as deep as CNNs. Adding skip connections and dilated convolutions to GCNs alleviates the training difficulty, which was impeding GCNs to go deeper and thus impeding further progress. A large number of experiments on semantic segmentation and part segmentation of 3D point clouds, as well as node classification on biological graphs show the benefit of deeper architectures, as they achieve state-of-the-art performance. We also show that our approach generalizes across several GCN operators.
For the point cloud tasks, we achieve the best results using EdgeConv \cite{wang2018understanding} as GCN operators for our backbone networks. Moreover, we find that dilated graph convolutions help to gain a larger receptive field without loss of resolution. Even with a small number of nearest neighbors, DeepGCNs can achieve high performance on point cloud semantic segmentation. \emph{ResGCN-112} and \emph{ResGCN-56} perform very well on this task, although they only use $4$ and $8$ nearest neighbors respectively compared to $16$ for \emph{ResGCN-28}. For the biological graph task, we achieve the best results using the MRGCN operator, which we propose as a novel memory-efficient alternative to EdgeConv. We successfully trained \emph{ResMRGCN-112} and \emph{DenseMRGCN-56}; both networks converged very well and achieved state-of-the-art results on the PPI dataset.
\section{Future Work}\label{sec:future}
Our results show that after solving the vanishing gradient problem plaguing deep GCNs, we can either make GCNs deeper or wider to get better performance. We expect GCNs to become a powerful tool for processing graph-structured data in computer vision, natural language processing, and data mining. We show successful cases for adapting concepts from CNNs to GCNs (\ie skip connections and dilated convolutions). In the future, it will be worthwhile to explore how to transfer other operators (\eg deformable convolutions \cite{dai2017deformable}), other architectures (\eg feature pyramid architectures \cite{zhao2017pyramid}), \etc. It will also be interesting to study different distance measures to compute dilated $k$-NN, constructing graphs with different $k$ at each layer, better dilation rate schedules \cite{chen2017rethinking, wang2018understanding} for GCNs, and combining residual and dense connections.
We also point out that, for the specific task of point cloud semantic segmentation, the common approach of processing the data in $1m\times 1m$ columns is sub-optimal for graph representation. A more suitable sampling approach should lead to further performance gains on this task. For the task of node classification, the existing datasets are relatively small. We expect that experimenting on larger datasets will further unleash the full potential of \emph{DeepGCNs}.
% use section* for acknowledgment
\ifCLASSOPTIONcompsoc
% The Computer Society usually uses the plural form
\section*{Acknowledgments}
\else
% regular IEEE prefers the singular form
\section*{Acknowledgment}
\fi
The authors thank Adel Bibi for his help with the project. This work was supported by the King Abdullah University of Science and Technology (KAUST) Office of Sponsored Research through the Visual Computing Center (VCC) funding.
%\appendices
\section{Qualitative Results for DeepGCNs} \label{appendix:qualitative_results}
Figures \ref{fig:qualitative_dilation}, \ref{fig:qualitative_nearest_neighbors}, \ref{fig:qualitative_layers}, \ref{fig:qualitative_filters}, \ref{fig:qualitative_56_28W} show qualitative results for DeepGCNs on S3DIS \cite{2017arXiv170201105A} and \figLabel \ref{fig:qualitative_partnet_appendix} shows qualitative results for DeepGCNs on PartNet \cite{Mo_2019_CVPR}.
\section{Run-time Overhead of Dynamic k-NN} \label{appendix:runtime_overhead}
We conduct a run-time experiment comparing the inference time of the reference model \emph{ResGCN-28} (28 layers, $k$=16) with dynamic k-NN and fixed k-NN. The inference time with fixed k-NN is 45.63ms. Computing the dynamic k-NN increases the inference time by 150.88ms. It is possible to reduce computation by updating the k-NN less frequently (\eg computing the dynamic k-NN every 3 layers).
\section{Comparison with DGCNN over All Classes} \label{appendix:dgcnn_all_classes}
To showcase the consistent improvement of our framework over the baseline DGCNN \cite{wang2018dynamic}, we reproduce the results of DGCNN\footnote{The results across all classes were not provided in the DGCNN paper.} in \tblLabel \ref{tbl:ours_vs_dgcnn} and find our method outperforms DGCNN in all classes.
\begin{table}[h]
\small
\centering
\setlength{\tabcolsep}{6pt}
\begin{tabular}{l|cc}
\toprule
\textbf{Class} & \textbf{DGCNN \cite{wang2018dynamic}} & \textbf{ResGCN-28 (\textit{Ours})} \\
\midrule
{ceiling} & 92.7 & \textbf{93.1} \\
{floor} & 93.6 & \textbf{95.3} \\
{wall} & 77.5 & \textbf{78.2} \\
{beam} & 32.0 & \textbf{33.9} \\
{column} & 36.3 & \textbf{37.4} \\
{window} & 52.5 & \textbf{56.1} \\
{door} & 63.7 & \textbf{68.2} \\
{table} & 61.1 & \textbf{64.9} \\
{chair} & 60.2 & \textbf{61.0} \\
{sofa} & 20.5 & \textbf{34.6} \\
{bookcase}& 47.7 & \textbf{51.5} \\
{board} & 42.7 & \textbf{51.1} \\
{clutter} & 51.5 & \textbf{54.4} \\
\midrule
\textbf{mIOU} & 56.3 & \textbf{60.0} \\
\bottomrule
\end{tabular}
\vspace{2pt}
\caption{\textbf{Comparison of \emph{ResGCN-28} with DGCNN}. Average per-class results across all areas for our reference network with 28 layers, \emph{residual graph connections} and \emph{dilated graph convolutions} compared to DGCNN baseline. \emph{ResGCN-28} outperforms DGCNN across all the classes. Metric shown is IoU.}
\label{tbl:ours_vs_dgcnn}
\end{table}
\begin{figure*}[!h]
\centering
\includegraphics[height=\textheight]{figures/dilation.png}
\caption{\textbf{Qualitative Results for S3DIS Semantic Segmentation}. We show the importance of stochastic dilated convolutions.}
\label{fig:qualitative_dilation}
\end{figure*}
\begin{figure*}[!h]
\centering
\includegraphics[height=\textheight]{figures/nearest_neighbor.png}
\caption{\textbf{Qualitative Results for S3DIS Semantic Segmentation}. We show the importance of the number of nearest neighbors used in the convolutions.}
\label{fig:qualitative_nearest_neighbors}
\end{figure*}
\begin{figure*}[!h]
\centering
\includegraphics[height=\textheight]{figures/layers.png}
\caption{\textbf{Qualitative Results for S3DIS Semantic Segmentation}. We show the importance of network depth (number of layers).}
\label{fig:qualitative_layers}
\end{figure*}
\begin{figure*}[!h]
\centering
\includegraphics[height=\textheight]{figures/filters.png}
\caption{\textbf{Qualitative Results for S3DIS Semantic Segmentation}. We show the importance of network width (number of filters per layer).}
\label{fig:qualitative_filters}
\end{figure*}
\begin{figure*}[!h]
\centering
\includegraphics[height=\textheight]{figures/56_28W.png}
\caption{\textbf{Qualitative Results for S3DIS Semantic Segmentation}. We show the benefit of a wider and deeper network even with only half the number of nearest neighbors.}
\label{fig:qualitative_56_28W}
\end{figure*}
\begin{figure*}[!h]
\centering
\includegraphics[page=1,trim = 0mm 0mm 200mm 300mm, clip, width=3.55in]{figures/Plain28Res28.pdf}
\includegraphics[page=3,trim = 0mm 0mm 200mm 300mm, clip, width=3.55in]{figures/Plain28Res28.pdf}
\includegraphics[page=5,trim = 0mm 0mm 200mm 600mm, clip, width=3.55in]{figures/Plain28Res28.pdf}
\includegraphics[page=2,trim = 0mm 0mm 200mm 600mm, clip, width=3.55in]{figures/Plain28Res28.pdf}
\includegraphics[page=4,trim = 0mm 0mm 200mm 600mm, clip, width=3.55in]{figures/Plain28Res28.pdf}
\includegraphics[page=6,trim = 0mm 0mm 200mm 600mm, clip, width=3.55in]{figures/Plain28Res28.pdf}
\includegraphics[page=7,trim = 0mm 0mm 200mm 600mm, clip, width=3.55in]{figures/Plain28Res28.pdf}
\includegraphics[page=8,trim = 0mm 0mm 200mm 600mm, clip, width=3.55in]{figures/Plain28Res28.pdf}
\includegraphics[page=9,trim = 0mm 0mm 200mm 600mm, clip, width=3.55in]{figures/Plain28Res28.pdf}
\includegraphics[page=10,trim = 0mm 0mm 200mm 600mm, clip, width=3.55in]{figures/Plain28Res28.pdf}
\includegraphics[page=11,trim = 0mm 0mm 200mm 600mm, clip, width=3.55in]{figures/Plain28Res28.pdf}
\includegraphics[page=12,trim = 0mm 0mm 200mm 600mm, clip, width=3.55in]{figures/Plain28Res28.pdf}
\includegraphics[page=13,trim = 0mm 0mm 200mm 600mm, clip, width=3.55in]{figures/Plain28Res28.pdf}
\includegraphics[page=14,trim = 0mm 0mm 200mm 600mm, clip, width=3.55in]{figures/Plain28Res28.pdf}
\includegraphics[page=15,trim = 0mm 0mm 200mm 600mm, clip, width=3.55in]{figures/Plain28Res28.pdf}
\includegraphics[page=16,trim = 0mm 0mm 200mm 600mm, clip, width=3.55in]{figures/Plain28Res28.pdf}
\includegraphics[page=17,trim = 0mm 0mm 200mm 600mm, clip, width=3.55in]{figures/Plain28Res28.pdf}
\includegraphics[page=1, trim = 0mm 0mm 200mm 600mm, clip, width=3.55in]{figures/empty.jpg}
\caption{\textbf{Qualitative Results on PartNet Part Segmentation}. We illustrate our performance compared to the ground truth on PartNet. }
\label{fig:qualitative_partnet_appendix}
\end{figure*}
% Generated by IEEEtran.bst, version: 1.14 (2015/08/26)
\begin{thebibliography}{10}
\providecommand{\url}[1]{#1}
\csname url@samestyle\endcsname
\providecommand{\newblock}{\relax}
\providecommand{\bibinfo}[2]{#2}
\providecommand{\BIBentrySTDinterwordspacing}{\spaceskip=0pt\relax}
\providecommand{\BIBentryALTinterwordstretchfactor}{4}
\providecommand{\BIBentryALTinterwordspacing}{\spaceskip=\fontdimen2\font plus
\BIBentryALTinterwordstretchfactor\fontdimen3\font minus
\fontdimen4\font\relax}
\providecommand{\BIBforeignlanguage}[2]{{%
\expandafter\ifx\csname l@#1\endcsname\relax
\typeout{** WARNING: IEEEtran.bst: No hyphenation pattern has been}%
\typeout{** loaded for the language `#1'. Using the pattern for}%
\typeout{** the default language instead.}%
\else
\language=\csname l@#1\endcsname
\fi
#2}}
\providecommand{\BIBdecl}{\relax}
\BIBdecl
\bibitem{social_tang2009relational}
L.~Tang and H.~Liu, ``Relational learning via latent social dimensions,'' in
\emph{Proceedings of the 15th ACM SIGKDD international conference on
Knowledge discovery and data mining}.\hskip 1em plus 0.5em minus 0.4em\relax
ACM, 2009, pp. 817--826.
\bibitem{chem_zitnik2017predicting}
M.~Zitnik and J.~Leskovec, ``Predicting multicellular function through
multi-layer tissue networks,'' \emph{Bioinformatics}, vol.~33, no.~14, pp.
i190--i198, 2017.
\bibitem{chem_wale2008comparison}
N.~Wale, I.~A. Watson, and G.~Karypis, ``Comparison of descriptor spaces for
chemical compound retrieval and classification,'' \emph{Knowledge and
Information Systems}, vol.~14, no.~3, pp. 347--375, 2008.
\bibitem{rec_monti2017geometric}
F.~Monti, M.~Bronstein, and X.~Bresson, ``Geometric matrix completion with
recurrent multi-graph neural networks,'' in \emph{Advances in Neural
Information Processing Systems}, 2017, pp. 3697--3707.
\bibitem{rec_ying2018graph}
R.~Ying, R.~He, K.~Chen, P.~Eksombatchai, W.~L. Hamilton, and J.~Leskovec,
``Graph convolutional neural networks for web-scale recommender systems,'' in
\emph{Proceedings of the 24th ACM SIGKDD International Conference on
Knowledge Discovery \& Data Mining}.\hskip 1em plus 0.5em minus 0.4em\relax
ACM, 2018, pp. 974--983.
\bibitem{wang2018dynamic}
Y.~Wang, Y.~Sun, Z.~Liu, S.~E. Sarma, M.~M. Bronstein, and J.~M. Solomon,
``Dynamic graph cnn for learning on point clouds,'' \emph{arXiv preprint
arXiv:1801.07829}, 2018.
\bibitem{li2018deeper}
Q.~Li, Z.~Han, and X.-M. Wu, ``Deeper insights into graph convolutional
networks for semi-supervised learning,'' in \emph{Thirty-Second AAAI
Conference on Artificial Intelligence}, 2018.
\bibitem{wu2019comprehensive}
Z.~Wu, S.~Pan, F.~Chen, G.~Long, C.~Zhang, and P.~S. Yu, ``A comprehensive
survey on graph neural networks,'' \emph{arXiv preprint arXiv:1901.00596},
2019.
\bibitem{zhou2018graph}
J.~Zhou, G.~Cui, Z.~Zhang, C.~Yang, Z.~Liu, and M.~Sun, ``Graph neural
networks: A review of methods and applications,'' \emph{arXiv preprint
arXiv:1812.08434}, 2018.
\bibitem{he2016deep}
K.~He, X.~Zhang, S.~Ren, and J.~Sun, ``Deep residual learning for image
recognition,'' in \emph{Proceedings of the IEEE conference on computer vision
and pattern recognition}, 2016, pp. 770--778.
\bibitem{huang2017densely}
G.~Huang, Z.~Liu, L.~Van Der~Maaten, and K.~Q. Weinberger, ``Densely connected
convolutional networks,'' in \emph{Proceedings of the IEEE conference on
computer vision and pattern recognition}, 2017, pp. 4700--4708.
\bibitem{yu2015multi}
F.~Yu and V.~Koltun, ``Multi-scale context aggregation by dilated
convolutions,'' \emph{arXiv preprint arXiv:1511.07122}, 2015.
\bibitem{2017arXiv170201105A}
I.~Armeni, S.~Sax, A.~R. Zamir, and S.~Savarese, ``{Joint 2D-3D-Semantic Data
for Indoor Scene Understanding},'' \emph{ArXiv e-prints}, Feb. 2017.
\bibitem{Mo_2019_CVPR}
K.~Mo, S.~Zhu, A.~X. Chang, L.~Yi, S.~Tripathi, L.~J. Guibas, and H.~Su,
``{PartNet}: A large-scale benchmark for fine-grained and hierarchical
part-level {3D} object understanding,'' in \emph{The IEEE Conference on
Computer Vision and Pattern Recognition (CVPR)}, June 2019.
\bibitem{li2019deepgcns}
G.~Li, M.~Müller, A.~Thabet, and B.~Ghanem, ``Deepgcns: Can gcns go as deep as
cnns?'' in \emph{The IEEE International Conference on Computer Vision
(ICCV)}, 2019.
\bibitem{nlp_bastings2017graph}
J.~Bastings, I.~Titov, W.~Aziz, D.~Marcheggiani, and K.~Simaan, ``Graph
convolutional encoders for syntax-aware neural machine translation,'' in
\emph{Proceedings of the 2017 Conference on Empirical Methods in Natural
Language Processing}, 2017, pp. 1957--1967.
\bibitem{nlp_marcheggiani2017encoding}
D.~Marcheggiani and I.~Titov, ``Encoding sentences with graph convolutional
networks for semantic role labeling,'' in \emph{Proceedings of the 2017
Conference on Empirical Methods in Natural Language Processing}, 2017, pp.
1506--1515.
\bibitem{qi20173d}
X.~Qi, R.~Liao, J.~Jia, S.~Fidler, and R.~Urtasun, ``3d graph neural networks
for rgbd semantic segmentation,'' in \emph{Proceedings of the IEEE
International Conference on Computer Vision}, 2017, pp. 5199--5208.
\bibitem{cv_scene_xu2017scene}
D.~Xu, Y.~Zhu, C.~B. Choy, and L.~Fei-Fei, ``Scene graph generation by
iterative message passing,'' in \emph{Proceedings of the IEEE Conference on
Computer Vision and Pattern Recognition}, 2017, pp. 5410--5419.
\bibitem{cv_scene_yang2018graph}
J.~Yang, J.~Lu, S.~Lee, D.~Batra, and D.~Parikh, ``Graph r-cnn for scene graph
generation,'' in \emph{Proceedings of the European Conference on Computer
Vision (ECCV)}, 2018, pp. 670--685.
\bibitem{cv_scene_li2018factorizable}
Y.~Li, W.~Ouyang, B.~Zhou, J.~Shi, C.~Zhang, and X.~Wang, ``Factorizable net:
an efficient subgraph-based framework for scene graph generation,'' in
\emph{Proceedings of the European Conference on Computer Vision (ECCV)},
2018, pp. 335--351.
\bibitem{cv_inv_scene_johnson2018image}
J.~Johnson, A.~Gupta, and L.~Fei-Fei, ``Image generation from scene graphs,''
in \emph{Proceedings of the IEEE Conference on Computer Vision and Pattern
Recognition}, 2018, pp. 1219--1228.
\bibitem{cv_action_yan2018spatial}
S.~Yan, Y.~Xiong, and D.~Lin, ``Spatial temporal graph convolutional networks
for skeleton-based action recognition,'' in \emph{Thirty-Second AAAI
Conference on Artificial Intelligence}, 2018.
\bibitem{cv_action_jain2016structural}
A.~Jain, A.~R. Zamir, S.~Savarese, and A.~Saxena, ``Structural-rnn: Deep
learning on spatio-temporal graphs,'' in \emph{Proceedings of the IEEE
Conference on Computer Vision and Pattern Recognition}, 2016, pp. 5308--5317.
\bibitem{mv_su2015multi}
H.~Su, S.~Maji, E.~Kalogerakis, and E.~Learned-Miller, ``Multi-view
convolutional neural networks for 3d shape recognition,'' in
\emph{Proceedings of the IEEE international conference on computer vision},
2015, pp. 945--953.
\bibitem{mv_guerry2017snapnet}
J.~Guerry, A.~Boulch, B.~Le~Saux, J.~Moras, A.~Plyer, and D.~Filliat,
``Snapnet-r: Consistent 3d multi-view semantic labeling for robotics,'' in
\emph{2017 IEEE International Conference on Computer Vision Workshops
(ICCVW)}, 2017, pp. 669--678.
\bibitem{mv_boulch2017unstructured}
A.~Boulch, B.~Le~Saux, and N.~Audebert, ``Unstructured point cloud semantic
labeling using deep segmentation networks.'' in \emph{3DOR}, 2017.
\bibitem{mv_li2016lstm}
Z.~Li, Y.~Gan, X.~Liang, Y.~Yu, H.~Cheng, and L.~Lin, ``Lstm-cf: Unifying
context modeling and fusion with lstms for rgb-d scene labeling,'' in
\emph{European Conference on Computer Vision}.\hskip 1em plus 0.5em minus
0.4em\relax Springer, 2016, pp. 541--557.
\bibitem{voxel_dai2017scannet}
A.~Dai, A.~X. Chang, M.~Savva, M.~Halber, T.~A. Funkhouser, and M.~Nie{\ss}ner,
``Scannet: Richly-annotated 3d reconstructions of indoor scenes.'' in
\emph{CVPR}, vol.~2, 2017, p.~10.
\bibitem{voxel_mv_qi2016volumetric}
C.~R. Qi, H.~Su, M.~Nie{\ss}ner, A.~Dai, M.~Yan, and L.~J. Guibas, ``Volumetric
and multi-view cnns for object classification on 3d data,'' in
\emph{Proceedings of the IEEE conference on computer vision and pattern
recognition}, 2016, pp. 5648--5656.
\bibitem{voxel_riegler2017octnet}
G.~Riegler, A.~O. Ulusoy, and A.~Geiger, ``Octnet: Learning deep 3d
representations at high resolutions,'' in \emph{Proceedings of the IEEE
Conference on Computer Vision and Pattern Recognition}, vol.~3, 2017.
\bibitem{voxel_tchapmi2017segcloud}
L.~Tchapmi, C.~Choy, I.~Armeni, J.~Gwak, and S.~Savarese, ``Segcloud: Semantic
segmentation of 3d point clouds,'' in \emph{3D Vision (3DV), 2017
International Conference on}.\hskip 1em plus 0.5em minus 0.4em\relax IEEE,
2017, pp. 537--547.
\bibitem{pc_qi2017pointnet}
C.~R. Qi, H.~Su, K.~Mo, and L.~J. Guibas, ``Pointnet: Deep learning on point
sets for 3d classification and segmentation,'' \emph{Proc. Computer Vision
and Pattern Recognition (CVPR), IEEE}, vol.~1, no.~2, p.~4, 2017.
\bibitem{pc_qi2017pointnet++}
C.~R. Qi, L.~Yi, H.~Su, and L.~J. Guibas, ``Pointnet++: Deep hierarchical
feature learning on point sets in a metric space,'' in \emph{Advances in
Neural Information Processing Systems}, 2017, pp. 5099--5108.
\bibitem{3dsemseg_ICCVW17}
F.~Engelmann, T.~Kontogianni, A.~Hermans, and B.~Leibe, ``Exploring spatial
context for 3d semantic segmentation of point clouds,'' in \emph{{IEEE}
International Conference on Computer Vision, 3DRMS Workshop, {ICCV}}, 2017.
\bibitem{pc_huang2018recurrent}
Q.~Huang, W.~Wang, and U.~Neumann, ``Recurrent slice networks for 3d
segmentation of point clouds,'' in \emph{Proceedings of the IEEE Conference
on Computer Vision and Pattern Recognition}, 2018, pp. 2626--2635.
\bibitem{pc_ye20183d}
X.~Ye, J.~Li, H.~Huang, L.~Du, and X.~Zhang, ``3d recurrent neural networks
with context fusion for point cloud semantic segmentation,'' in
\emph{European Conference on Computer Vision}.\hskip 1em plus 0.5em minus
0.4em\relax Springer, 2018, pp. 415--430.
\bibitem{kipf2016semi}
T.~N. Kipf and M.~Welling, ``Semi-supervised classification with graph
convolutional networks,'' \emph{arXiv preprint arXiv:1609.02907}, 2016.
\bibitem{pham2017column}
T.~Pham, T.~Tran, D.~Phung, and S.~Venkatesh, ``Column networks for collective
classification,'' in \emph{Thirty-First AAAI Conference on Artificial
Intelligence}, 2017.
\bibitem{rahimi2018semi}
A.~Rahimi, T.~Cohn, and T.~Baldwin, ``Semi-supervised user geolocation via
graph convolutional networks,'' \emph{arXiv preprint arXiv:1804.08049}, 2018.
\bibitem{xu2018representation}
K.~Xu, C.~Li, Y.~Tian, T.~Sonobe, K.-i. Kawarabayashi, and S.~Jegelka,
``Representation learning on graphs with jumping knowledge networks,''
\emph{arXiv preprint arXiv:1806.03536}, 2018.
\bibitem{hamilton2017inductive}
W.~Hamilton, Z.~Ying, and J.~Leskovec, ``Inductive representation learning on
large graphs,'' in \emph{Advances in Neural Information Processing Systems},
2017, pp. 1024--1034.
\bibitem{velivckovic2017graph}
P.~Veli{\v{c}}kovi{\'c}, G.~Cucurull, A.~Casanova, A.~Romero, P.~Lio, and
Y.~Bengio, ``Graph attention networks,'' \emph{arXiv preprint
arXiv:1710.10903}, 2017.
\bibitem{peng2017cross}
N.~Peng, H.~Poon, C.~Quirk, K.~Toutanova, and W.-t. Yih, ``Cross-sentence n-ary
relation extraction with graph lstms,'' \emph{Transactions of the Association
for Computational Linguistics}, vol.~5, pp. 101--115, 2017.
\bibitem{duvenaud2015convolutional}
D.~K. Duvenaud, D.~Maclaurin, J.~Iparraguirre, R.~Bombarell, T.~Hirzel,
A.~Aspuru-Guzik, and R.~P. Adams, ``Convolutional networks on graphs for
learning molecular fingerprints,'' in \emph{Advances in neural information
processing systems}, 2015, pp. 2224--2232.
\bibitem{li2015gated}
Y.~Li, D.~Tarlow, M.~Brockschmidt, and R.~Zemel, ``Gated graph sequence neural
networks,'' \emph{arXiv preprint arXiv:1511.05493}, 2015.
\bibitem{ioffe2015batch}
S.~Ioffe and C.~Szegedy, ``Batch normalization: Accelerating deep network
training by reducing internal covariate shift,'' \emph{arXiv preprint
arXiv:1502.03167}, 2015.
\bibitem{simonovsky2017dynamic}
M.~Simonovsky and N.~Komodakis, ``Dynamic edge-conditioned filters in
convolutional neural networks on graphs,'' in \emph{Proceedings of the IEEE
Conference on Computer Vision and Pattern Recognition}, 2017, pp. 3693--3702.
\bibitem{valsesia2018learning}
D.~Valsesia, G.~Fracastoro, and E.~Magli, ``Learning localized generative
models for 3d point clouds via graph convolution,'' 2018.
\bibitem{holschneider1990real}
M.~Holschneider, R.~Kronland-Martinet, J.~Morlet, and P.~Tchamitchian, ``A
real-time algorithm for signal analysis with the help of the wavelet
transform,'' in \emph{Wavelets}.\hskip 1em plus 0.5em minus 0.4em\relax
Springer, 1990, pp. 286--297.
\bibitem{shensa1992discrete}
M.~J. Shensa, ``The discrete wavelet transform: wedding the a trous and mallat
algorithms,'' \emph{IEEE Transactions on signal processing}, vol.~40, no.~10,
pp. 2464--2482, 1992.
\bibitem{xu2018powerful}
K.~Xu, W.~Hu, J.~Leskovec, and S.~Jegelka, ``How powerful are graph neural
networks?'' \emph{arXiv preprint arXiv:1810.00826}, 2018.
\bibitem{xu2018spidercnn}
Y.~Xu, T.~Fan, M.~Xu, L.~Zeng, and Y.~Qiao, ``Spidercnn: Deep learning on point
sets with parameterized convolutional filters,'' \emph{arXiv preprint
arXiv:1803.11527}, 2018.
\bibitem{Li2018PointCNNCO}
Y.~Li, R.~Bu, M.~Sun, W.~Wu, X.~Di, and B.~Chen, ``Pointcnn: Convolution on
x-transformed points,'' in \emph{NeurIPS}, 2018.
\bibitem{ben20183dmfv}
Y.~Ben-Shabat, M.~Lindenbaum, and A.~Fischer, ``3dmfv: Three-dimensional point
cloud classification in real-time using convolutional neural networks,''
\emph{IEEE Robotics and Automation Letters}, vol.~3, no.~4, pp. 3145--3152,
2018.
\bibitem{Thomas2019KPConvFA}
H.~Thomas, C.~R. Qi, J.-E. Deschaud, B.~Marcotegui, F.~Goulette, and L.~J.
Guibas, ``Kpconv: Flexible and deformable convolution for point clouds,''
\emph{2019 IEEE/CVF International Conference on Computer Vision (ICCV)}, pp.
6410--6419, 2019.
\bibitem{Li2018SONetSN}
J.~Li, B.~M. Chen, and G.~H. Lee, ``So-net: Self-organizing network for point
cloud analysis,'' \emph{2018 IEEE/CVF Conference on Computer Vision and
Pattern Recognition}, pp. 9397--9406, 2018.
\bibitem{Atzmon2018PointCN}
M.~Atzmon, H.~Maron, and Y.~Lipman, ``Point convolutional neural networks by
extension operators,'' \emph{ACM Transactions on Graphics (TOG)}, vol.~37,
pp. 1 -- 12, 2018.
\bibitem{Wang2018LocalSG}
C.~Wang, B.~Samari, and K.~Siddiqi, ``Local spectral graph convolution for
point set feature learning,'' \emph{ArXiv}, vol. abs/1803.05827, 2018.
\bibitem{Hermosilla2018MonteCC}
P.~Hermosilla, T.~Ritschel, P.-P. V{\'a}zquez, A.~Vinacua, and T.~Ropinski,
``Monte carlo convolution for learning on non-uniformly sampled point
clouds,'' \emph{ACM Transactions on Graphics (TOG)}, vol.~37, pp. 1 -- 12,
2018.
\bibitem{Groh2018FlexConvolutionM}
F.~Groh, P.~Wieschollek, and H.~P.~A. Lensch, ``Flex-convolution -
million-scale point-cloud learning beyond grid-worlds,'' in \emph{ACCV},
2018.
\bibitem{Fey/Lenssen/2019}
M.~Fey and J.~E. Lenssen, ``Fast graph representation learning with {PyTorch
Geometric},'' in \emph{ICLR Workshop on Representation Learning on Graphs and
Manifolds}, 2019.
\bibitem{chiang2019cluster}
W.-L. Chiang, X.~Liu, S.~Si, Y.~Li, S.~Bengio, and C.-J. Hsieh, ``Cluster-gcn:
An efficient algorithm for training deep and large graph convolutional
networks,'' \emph{arXiv preprint arXiv:1905.07953}, 2019.
\bibitem{chen2017vrgcn}
J.~Chen, J.~Zhu, and L.~Song, ``Stochastic training of graph convolutional
networks with variance reduction,'' \emph{arXiv preprint arXiv:1710.10568},
2017.
\bibitem{zhang2018gaan}
J.~Zhang, X.~Shi, J.~Xie, H.~Ma, I.~King, and D.-Y. Yeung, ``Gaan: Gated
attention networks for learning on large and spatiotemporal graphs,''
\emph{arXiv preprint arXiv:1803.07294}, 2018.
\bibitem{liu2019geniepath}
Z.~Liu, C.~Chen, L.~Li, J.~Zhou, X.~Li, L.~Song, and Y.~Qi, ``Geniepath: Graph
neural networks with adaptive receptive paths,'' in \emph{Proceedings of the
AAAI Conference on Artificial Intelligence}, vol.~33, 2019, pp. 4424--4431.
\bibitem{wang2018understanding}
P.~Wang, P.~Chen, Y.~Yuan, D.~Liu, Z.~Huang, X.~Hou, and G.~Cottrell,
``Understanding convolution for semantic segmentation,'' in \emph{2018 IEEE
Winter Conference on Applications of Computer Vision (WACV)}.\hskip 1em plus
0.5em minus 0.4em\relax IEEE, 2018, pp. 1451--1460.
\bibitem{dai2017deformable}
J.~Dai, H.~Qi, Y.~Xiong, Y.~Li, G.~Zhang, H.~Hu, and Y.~Wei, ``Deformable
convolutional networks,'' in \emph{Proceedings of the IEEE international
conference on computer vision}, 2017, pp. 764--773.
\bibitem{zhao2017pyramid}
H.~Zhao, J.~Shi, X.~Qi, X.~Wang, and J.~Jia, ``Pyramid scene parsing network,''
in \emph{Proceedings of the IEEE conference on computer vision and pattern
recognition}, 2017, pp. 2881--2890.
\bibitem{chen2017rethinking}
L.-C. Chen, G.~Papandreou, F.~Schroff, and H.~Adam, ``Rethinking atrous
convolution for semantic image segmentation,'' \emph{arXiv preprint
arXiv:1706.05587}, 2017.
\end{thebibliography}