Vanilla ViT for Automotive Point Cloud Semantic Segmentation
Extracted source text · arXiv 2605.31177
\documentclass[10pt,twocolumn,letterpaper]{article}
%%%%%%%%% PAPER TYPE - PLEASE UPDATE FOR FINAL VERSION
\usepackage{cvpr} % To produce the CAMERA-READY version
% \usepackage[review]{cvpr} % To produce the REVIEW version
% \usepackage[pagenumbers]{cvpr} % To force page numbers, e.g. for an arXiv version
% It is strongly recommended to use hyperref, especially for the review version.
% hyperref with option pagebackref eases the reviewers' job.
% Please disable hyperref *only* if you encounter grave issues,
% e.g. with the file validation for the camera-ready version.
%
% If you comment hyperref and then uncomment it, you should delete *.aux before re-running LaTeX.
% (Or just hit 'q' on the first LaTeX run, let it finish, and you should be clear).
\usepackage[table, dvipsnames]{xcolor}
\definecolor{cvprblue}{rgb}{0.21,0.49,0.74}
\usepackage[pagebackref,breaklinks,colorlinks,citecolor=cvprblue]{hyperref}
\usepackage[capitalize]{cleveref}
% Include other packages here, before hyperref.
\usepackage{pifont}
\usepackage{graphicx}
\usepackage{booktabs}
\usepackage{multirow}
\usepackage{adjustbox}
\usepackage{array}
\newcolumntype{R}[2]{%
>{\adjustbox{angle=#1,lap=1.3\width-(#2)}\bgroup}%
l%
<{\egroup}%
}
\newcommand{\ra}[1]{\renewcommand{\arraystretch}{#1}}
% The "axessiblity" package can be found at: https://ctan.org/pkg/axessibility?lang=en
\usepackage[accsupp]{axessibility} % Improves PDF readability for those with disabilities.
% ---------------------------------------------------------------
% Hyperref package
% It is strongly recommended to use hyperref, especially for the review version.
% Please disable hyperref *only* if you encounter grave issues.
% hyperref with option pagebackref eases the reviewers' job, but should be disabled for the final version.
%
% If you comment hyperref and then uncomment it, you should delete
% main.aux before re-running LaTeX.
% (Or just hit 'q' on the first LaTeX run, let it finish, and you
% should be clear).
% TODO FINAL: Comment out the following line for the camera-ready version
%\usepackage[pagebackref,breaklinks,colorlinks,citecolor=eccvblue]{hyperref}
% TODO FINAL: Un-comment the following line for the camera-ready version
% \usepackage[pagebackref]{hyperref}
% Support for ORCID icon
\usepackage{orcidlink}
\definecolor{turquoise}{rgb}{0.20, 0.70, 0.65}
\def\pmixplus{PillarMix\textsuperscript{\smash+}\xspace}
\definecolor{better}{rgb}{0.19, 0.55, 0.91}
\definecolor{worse}{rgb}{0.82, 0.1, 0.26}
\newcommand{\cmark}{\textcolor{better}{\ding{51}}}
\newcommand{\xmark}{\textcolor{worse}{\ding{55}}}
% ---------------------------------------------------------------
% TODO REVIEW: Replace with your title
\title{Vanilla ViT for Automotive Point Cloud Semantic Segmentation}
% TODO REVIEW: If the paper title is too long for the running head, you can set
% an abbreviated paper title here. If not, comment out.
% \titlerunning{VaViT}
\author{
% --- Line 1 ---
Gilles Puy\textsuperscript{1} \quad
Nermin Samet\textsuperscript{1} \quad
Alexandre Boulch\textsuperscript{1} \\
Spyros Gidaris\textsuperscript{1} \quad
Tuan-Hung VU\textsuperscript{1} \quad
Renaud Marlet\textsuperscript{1,2}
% --- Spacing ---
\vspace{0.3cm} \\
% --- Institution ---
\textsuperscript{1}valeo.ai, Paris, France \quad
\textsuperscript{2}LIGM, CNRS, Univ Gustave Eiffel, ENPC, IP Paris, France \\
% --- Emails ---
% {\tt\small \{firstname.lastname\}@valeo.com}
}
\begin{document}
\maketitle
\begin{abstract}
Plain Transformers have become the de-facto architecture for processing text, audio, image, and video, offering a unified backbone for multimodal learning. However, state-of-the-art architectures for point cloud semantic segmentation remain dominated by U-Nets architectures where convolutions are interleaved with local or windowed attentions. In this work, we show how to effectively leverage vanilla, non-hierarchical ViTs for segmentation of large-scale automotive lidar scenes. We bridge the performance gap thanks to a carefully designed tokenizer, a lightweight decoder segmentation head, and tailored data augmentations. Our approach, VaViT for Vanilla ViT, matches or exceeds the performance of state-of-the-art methods while maintaining the simplicity of ViT architecture. We provide extensive evaluations on nuScenes, SemanticKITTI, and Waymo Open Dataset to validate the efficiency of our method. Code and models are available at \url{https://github.com/valeoai/VaViT}.
\end{abstract}
% ===============================================
%
% ===============================================
\section{Introduction}
\label{sec:intro}
Transformers have become the dominant architecture across modalities, powering state-of-the-art systems in image \cite{dosovitskiy2021image}, language \cite{vaswani2017attention}, audio \cite{gong2021ast}, and video \cite{arnab2021vivit} processing. Their uniform design, based on self-attention and feed-forward blocks, enables a shared modeling paradigm and greatly facilitates multimodal learning \cite{xu2023multimodal}. Vision Transformers (ViTs), in particular, have demonstrated remarkable scaling properties: while Convolutional Neural Networks (CNNs) benefit from strong inductive biases and perform well in limited-data regimes, ViTs exhibit superior power-law scaling and continue to improve as model size and dataset scale increase \cite{dosovitskiy2021image, zhai2022scaling, dehghani2023scaling}. This ability to learn global dependencies without relying on handcrafted architectural priors makes plain Transformers an appealing candidate for a unified backbone across perception tasks.
However, this architectural simplicity has not translated to point cloud semantic segmentation. Current state-of-the-art methods depart significantly from vanilla Transformers.
A first line of work, exemplified by PTv3 \cite{ptv3} and the recent LitePT \cite{litept}, adopts hybrid U-Net-style architectures in which sparse convolutions are interleaved with local self-attention, moving away from the plain Transformer paradigm. Moreover, empirical evidence from LitePT demonstrates that most of the “heavy lifting” is performed by the convolutional layers, questioning if they could be replaced by generic global self-attention.
Such hybridization prevents a clean unification with Transformer-based architectures used in images, audio, or text, and limits the benefits of architectural standardization.
A second line of work, including RangeViT \cite{rangevit} and RangeFormer \cite{rangeformer}, embraces a more standard Transformer backbone but operates on 2D range-view projections of lidar data. While computationally convenient, this projection inevitably distorts the underlying 3D geometry and discards part of the spatial structure. As a result, these approaches remain constrained by the representation bottleneck induced by the 2D view and generally underperform methods that reason directly on raw 3D points.
In this work, we ask a simple question: can a plain, non-hierarchical Vision Transformer operate directly on 3D point clouds and achieve competitive performance for large-scale lidar semantic segmentation? We answer this question positively by introducing VaViT, a point cloud segmentation framework that processes raw 3D points and leverages a \underline{va}nilla \underline{ViT} as its core backbone.
%
Rather than using U-Net-style backbones combining sparse convolutions and local attention, our approach employs a single-scale vanilla ViT backbone with global self-attention and without hierarchical stages or convolutional blocks. To that end, it connects point clouds directly to the ViT via a carefully designed interface.
The success of this design relies on three key ingredients. First, we introduce a dedicated tokenization strategy. In contrast to range-view approaches that convert the entire lidar scan into a 2D image representation, we extract point-wise embeddings directly in 3D space and project them onto a coarse Bird’s Eye View (BEV) grid to form tokens.
Despite this coarse projection, we show that a properly designed tokenization stem preserves sufficient geometric information for downstream semantic segmentation. These tokens are then processed by a standard ViT, without architectural modifications. Second, we design a lightweight decoder head that combines the global contextual features produced by the ViT with the high-resolution point embeddings computed prior to tokenization, enabling precise semantic labeling while keeping the overall architecture simple and efficient. Third, we demonstrate that training a vanilla ViT effectively on autonomous driving datasets requires tailored data augmentation strategies. Compared to the massive datasets typically used to pretrain ViTs, lidar semantic segmentation benchmarks are relatively limited in scale. We therefore introduce strong geometric augmentations, including the generation of composite scenes by mixing portions of different scans. In particular, we show that mixing pillars in BEV space significantly enhances generalization.
Through extensive experiments on nuScenes \cite{nuscenes}, SemanticKITTI \cite{semkitti}, and Waymo Open Dataset \cite{waymo}, we demonstrate that a carefully designed vanilla ViT pipeline can match or surpass state-of-the-art methods for lidar semantic segmentation, while offering a simpler and more unified architecture. Our results suggest that specialized hybrid designs in point cloud segmentation migth not be required, and that plain Transformers constitute a viable alternative for 3D semantic segmentation.
In summary, our contributions are the following.
\begin{itemize}
\item We show that using a vanilla ViT can match or surpass state-of-the-art backbones for semantic segmentation of lidar point clouds.
\item To that end, we introduce a dedicated tokenization strategy to feed the ViT with sufficiently informative tokens computed on a coarse grid in BEV. We also define a lightweight segmentation head.
\item We introduce a strong geometric augmentations strategy called \pmixplus, which enhances generalization of our VaViT models.
\end{itemize}
% ===============================================
%
% ===============================================
\section{Related works}
\label{sec:related}
\subsubsection{Network architectures on points.} There are four broad types of network architectures to process point clouds: (1)~directly working on points, (2)~working on projected versions of the point clouds, (3)~working on voxelized representations, (4)~mixing several types of representations.
PointNet \cite{pointnet} and PointNet++ \cite{pointnetpp}, which work directly on points, pioneered the field and mainly consists of a repetition of shared MLPs applied pointwise. Several improvements have followed, by defining point convolution operators, e.g., \cite{kpconv,fkaconv}, leveraging networks developed for graph--structered data, e.g., \cite{dgcnn,SPGraph}, improving the training recipe \cite{pointnext}, or taking inspiration from architectures developed for images such as the MLP Mixer \cite{mlpmixer}, e.g., \cite{pointmixer,pointmlp,waffleiron}.
Among the second type of architectures, the most used projection is probably the range projection \cite{rangenetpp,salsaNext,SqueezeSegV3,SCSSnet,KPRNet}, which is a natural representation to use for rotating lidar. Some other work leverages the projection in bird's-eye view (BEV) \cite{Polarnet,lmscnet}.
The invention and release of fast implementations for sparse convolutions on GPU has greatly facilitated work on voxel representations of large-scale point clouds \cite{minknet,spvnas}. The basic architectures were then improved by adapting the shape of the voxels to the structure of lidar point clouds \cite{Cylinder3d}. Self-distillation techniques was also used to improve performance \cite{PVKD,sdseg3d}. Attention mechanisms, added on top of such convolutions, also helped in reaching better scores \cite{AF2S3Net,gasn,svaseg,sphereformer}. Such convolutions are still today part of state-of-the-art architectures \cite{ptv3,litept}.
Finally, several works have shown that it can be advantageous to fuse some of the representations described above \cite{tornadonet,amvnet,gfnet,fusionnet,pcscnet,RPVNet}.
\subsubsection{Transformer on points.}
In the context of large scale 3D point cloud processing,
directly using self-attention layers on all points is hardly feasible due to the large number of points and the quadratic complexity of attention.
%
To maintain computational efficiency while still capturing complex geometric structures, early methods like the Point Transformer \cite{pointtransformer} applied self-attentions locally and relied on farthest point sampling to downsample the point cloud and be able to capture larger scale structures.
%
Fast Point Transformer \cite{fastpointtrans} proposed to combine local attention with a voxel-based hashing technique to reach much faster inference time.
%
Stratified Transformer \cite{strattrans} then introduced a key
strategy that samples nearby points densely and distant points sparsely, enlarging the receptive field in the attention layers while keeping a low computational cost.
%
Superpoint Transformer \cite{superpointtransformer} proposed to construct a hierarchical superpoint structure and to capture the relationship between superpoints using a self-attention mechanism.
%
SphereFormer \cite{sphereformer} proposed the use of radial window self-attention to aggregate information from close and distant points from the sensor.
%
PTv3 \cite{ptv3} transitioned toward high-efficiency serialization, using space-filling curves to reorder points into a 1D sequence, allowing for fast grouping via linear sequence chunks rather than exhaustive spatial searches.
%
Finally, LitePT \cite{litept} improved upon PTv3 with a lighter and better performing architecture that uses sparse convolutions in the early and last layers of the U-Net architecture and self-attentions in the the middle layers.
In parallel to point approaches, projection-based methods have simplified tokenization by working in 2D and leveraging the regular 2D grid structure. RangeViT \cite{rangevit} and RangeFormer \cite{rangeformer} both used range projection, which is tokenized % and tokenization
by extracting small patches. FlatFormer \cite{flatformer} introduced a flattened window attention mechanism that tokenizes the point cloud by partitioning and flattening it into group of equal sizes. FlatFormer was originally tested for object detection and applied on a BEV representation of the point clouds. FlatFormer and VaViT thus share the same type of input representation. However their downstream architecture is completely different: a regular ViT for us, with global self-attention; local attentions within groups constructed by alternating the sorting spatial dimension.
\subsubsection{Lidar data augmentations.}
Beyond standard augmentations such as rotation and global scaling, it has been shown that mixing-based strategies can greatly improve the performance of lidar point cloud segmentation.
%
Instance CutMix \cite{RPVNet} improves the segmentation performance of rare objects by extracting such object instances from a source point cloud and pasting them into a target scene. %
However, simply inserting such objects in a scan breaks the lidar scan pattern.
%
To better preserve this pattern, LaserMix \cite{lasermix} partition scans into non-overlapping areas along the inclination angle and then create a new scan by mixing these partitions.
%
PolarMix \cite{polarmix} does a similar operation but with non-overlapping partitions done along the azimuth angle.
%
Finally, complementing these partitioning schemes, PillarMix \cite{pillarmix} utilizes a pillar-based mixing approach to generate augmented training samples. We propose an improved version, \pmixplus, for training our ViT backbones.
% ===============================================
%
% ===============================================
\section{Method}
\label{sec:method}
Our goal is to perform point cloud semantic segmentation using a plain Vision Transformer while operating directly on raw 3D points. To this end, we design a pipeline that bridges the gap between irregular point cloud data and the token-based representation expected by Transformers, while preserving fine geometric information for point-level prediction. An overview of the proposed framework is illustrated in \cref{fig:vavit}. We first introduce a tokenizer that converts raw point clouds into a set of pillar tokens defined in BEV while extracting expressive point-level embeddings (\cref{sec:tokenizer}). These tokens are then processed by a standard, non-hierarchical Vision Transformer that models long-range spatial dependencies across the scene (\cref{sec:method_vit}). To recover fine-grained predictions, we design a lightweight segmentation head that redistributes the transformer tokens to individual points and combines them with the original point embeddings (\cref{sec:method_head}). Finally, we introduce a data augmentation strategy tailored to lidar segmentation, based on mixing pillars from multiple scans, which improves generalization when training vanilla ViTs on autonomous driving datasets (\cref{sec:method_aug}).
\begin{figure*}[t]
\centering
\includegraphics[width=0.9\linewidth]{images/VaViT.png}
\caption{\textbf{Overview.} Our tokenizer aggregates point-level embeddings $\mathbf{p}_i$ into $Q$ non-empty pillar embeddings, which serves as input tokens $\mathbf{t}_q$ for a vanilla Vision Transformer. After being processed by the ViT, the $Q$ pillar tokens are redistributed
and merged with the original point embeddings, forming the final representations used for point classification. Positions on the 2D BEV space are encoded using RoPE.}
\label{fig:vavit}
\end{figure*}
% ===============================================
\subsection{Tokenizer}
\label{sec:tokenizer}
The tokenizer takes as input a list of $N$ points, each associated with a low dimensional features (typically $x$, $y$, $z$ coordinates, range and intensity). These point features passes in a embedding network to provide high-dimensional point embeddings. The architecture of this embedding network is inspired from the embedding layer used in \cite{waffleiron} and consists of a series of shared PointNet++/DGCNN-like layers~\cite{pointnet,dgcnn}, as described below.
%
The token given as input to the ViT are then computed in BEV: one token per square pillar of a preset side dimension.
For a given pillar, we gather the embeddings of all points falling into this pillar and the corresponding pillar token is computed by max-pooling over this set of embeddings.
More formally, let us denote by $\mathbf{p}_i^{(\ell)} \in \mathbb{R}^{F_\ell}$, % \thvc{why $N \times$?},
the features of the $i^{\rm th}$ point at the $\ell^{\rm th}$ layer of the tokenizer. The tokenizer consists of a repetition of $L_{\rm emb}$ layers $f^{(\ell)}(\cdot)$ applied pointwise. The computation of features $\mathbf{p}_i^{(\ell+1)}$ is decomposed into three stages: a first stage working directly on the point feature $\mathbf{p}_i^{(\ell)}$; a second stage operating on the local representations $\mathbf{p}^{(\ell)}_j - \mathbf{p}^{(\ell)}_i$, with $j \in \mathcal{N}_i$ and where $\mathcal{N}_i$ is the set of $k$-nearest neighbors (in Euclidean space) to the $i^{\rm th}$ point; a third stage combining the results of the first two.
The first stage produces an intermediate point representation obtained by applying to $\mathbf{p}^{(\ell)}_i$ a linear layer, followed by a batchnorm layer (BN) and a ReLU activation. This layer is denoted by $f_{\rm glob}^{(\ell)}(\cdot)$
For the second stage, at a local level, we start by computing all the relative representations $\tilde{\mathbf{p}}^{(\ell)}_{ij} = f_{\rm rel}^{(\ell)}(\mathbf{p}^{(\ell)}_j - \mathbf{p}^{(\ell)}_i)$ and obtain the local representation for point $i$ as
%
\begin{align}
\tilde{\mathbf{p}}^{(\ell)}_{i} = \max_{j \in \mathcal{N}_i}
\left(
f_{\rm gate}^{(\ell)} ( \tilde{\mathbf{p}}^{(\ell)}_{ij} )
\odot
\tilde{\mathbf{p}}^{(\ell)}_{ij}
\right),
\end{align}
%
where $f_{\rm rel}^{(\ell)}(\cdot)$ has the same architecture as $f_{\rm glob}^{(\ell)}(\cdot)$, i.e., a linear layer followed by BN and ReLU, $\odot$ denotes the channelwise multiplication, and $\max$ is applied channelwise (max pooling). To improve the quality of these point representations for downstream processing by the ViT, we propose to use a gating mechanism $f_{\rm gate}(\cdot)$, which consists of a linear layer followed by a sigmoid, to filter out spurious features in the set of neighbors $\mathcal{N}_i$.
The third stage, which produces the final representation of point $i$ at layer $\ell + 1$, then satisfies
%
\begin{align}
\mathbf{p}^{(\ell + 1)}_i
=
f_{\rm cat}^{(\ell)}
\left(
\left[
f_{\rm glob}^{(\ell)}(\mathbf{p}^{(\ell)}_i)
\; , \;
\tilde{\mathbf{p}}^{(\ell)}_{i}
\right]
\right),
\end{align}
%
where $f_{\rm cat}^{(\ell)}(\cdot)$ is just a linear layer and $[\cdot , \cdot]$ denotes the concatenation. This process is repeated $L_{\rm emb}$ times to obtain $\mathbf{p}^{(L_{\rm emb})}_i$. In our design, $F_{\ell} = F_{\ell+1}$ except for $F_0$ and $F_{L_{\rm emb}}$ which matches the size of the input point features and the inner dimension of the ViT, respectively.
The initial tokens $\mathbf{t}_q^{(0)}$, $q = 1, \ldots, Q$, provided as input to the ViT, are obtained by dividing the scene into regular non-overlapping square pillars of side length~$s$. For each \emph{non-empty} pillar~$q$, the representations $\mathbf{p}^{(L_{\rm emb})}_i$ of all points falling into the square are gathered, and a max pooling operation is performed on these representations to obtain a pillar embedding $\mathbf{t}_q^{(0)}$
that we use as initial pillar token.
Note that we compute a token only for occupied pillars, so in practice the number of tokens $Q$ varies per point cloud. Each token $\mathbf{t}_q$ is associated with the 2D coordinates $\mathbf{c}_q$ of the center of its corresponding pillar~$q$.
% ===============================================
\subsection{Vanilla ViT}
\label{sec:method_vit}
The tokens $\mathbf{t}_q^{(0)}$, $q = 1, \ldots, Q$, are processed with a Vanilla ViT \cite{dosovitskiy2021image}:
\begin{align}
\tilde{\mathbf{T}}^{(\ell + 1)}
& = {\rm MSA}( {\rm LN}( \mathbf{T}^{(\ell)} ), \; \mathbf{C}) + \mathbf{T}^{(\ell)}
&\quad \ell = 0, \ldots, L_{\rm vit} - 1
\\
\mathbf{T}^{(\ell + 1)}
& = {\rm MLP}( {\rm LN}( \tilde{\mathbf{T}}^{(\ell + 1)} ) ) + \tilde{\mathbf{T}}^{(\ell + 1)}
&\quad \ell = 0, \ldots, L_{\rm vit} - 1
\end{align}
where $\mathbf{T}^{(0)} = [\mathbf{t}_1^{(0)}, \ldots, \mathbf{t}_Q^{(0)}]$ and $\mathbf{C} = [\mathbf{c}_1, \ldots, \mathbf{c}_Q]$, and $L_{\rm vit}$ is the number of transformer blocks.
%
Layernorm (LN) is applied before the multi-head self-attention (MSA) and the MLP.
For positional encoding in the MSA, we use a classical 2D RoPE by repeating the 1D RoPE operation on each axis \cite{rope,heo2024ropevit}.
%
The MLP contains two layers with a GELU non-linearity.
To allow batch processing of multiple point clouds, the token sequence length for each point cloud is equalized to the maximum length (maximum number of non-empty pillars for a point cloud) in a batch by repeating a padding token the appropriate number of times.
% ===============================================
\subsection{Segmentation head}
\label{sec:method_head}
To allow semantic segmentation at the level of the points, the tokens $\mathbf{T}^{( L_{\rm vit})} = [\mathbf{t}_1^{( L_{\rm vit})}, \ldots, \mathbf{t}_Q^{( L_{\rm vit})}]$ are lifted from BEV to 3D: each point in a given pillar $q$ receives the corresponding token $\mathbf{t}_q^{( L_{\rm vit})}$. This operation alone would be insufficient to differentiate the individual classes of the points falling into the same pillar. Each lifted feature vector is thus augmented with the corresponding point embedding $\mathbf{p}^{(L_{\rm emb})}_i$.
With a slight abuse of notations, let $\mathbf{t}_i^{( L_{\rm vit})}$ be the features lifted from BEV for point $i$. The final features $\mathbf{r}_i$ for point $i$ satisfies
%
\begin{align}
\tilde{\mathbf{r}}_i
& =
[g_{\rm vit}(\mathbf{t}^{(L_{\rm vit})}_i) \; , \; g_{\rm emb}(\mathbf{p}^{(L_{\rm emb})}_i)],
\\
\mathbf{r}_i
& =
g_{\rm final}\left( g_{\rm gate}(\tilde{\mathbf{r}}_i) \odot \tilde{\mathbf{r}}_i \right),
\end{align}
%
where $g_{\rm vit}$, $g_{\rm emb}$, $g_{\rm final}$ contain a linear layer followed by batchnorm and ReLU, and have an output dimension set to the inner dimension $F_\ell$ of the tokenizer. Depending on the type of object to classify, it might be beneficial, to make a decision, to adjust the focus more on the initial point embedding or on the representation coming out of the ViT.
We therefore us a gating function $g_{\rm gate}$, made of a linear layer followed by a sigmoid, to reweight the contribution of each channel of the two types of representations.
The final point feature $\mathbf{r}_i$ then enters a linear classification layer in charge of classifying the point into one of the semantic classes.
% ===============================================
\subsection{Point cloud augmentation with \pmixplus}
\label{sec:method_aug}
Transformers in general, and ViTs in particular, may need large datasets to train well \cite{dosovitskiy2021image}. Data augmentation is thus a major strategy for improving performance, especially in the automotive lidar setting where the number of point clouds is often considerably smaller than the amount of images used to train ViTs.
Classical point cloud augmentations include random rotation, random flipping and random scaling. To further boost the performance of VaViT, we introduce an augmentation called \pmixplus.
This point cloud transformation is an improvement of PillarMix, which was used in \cite{pillarmix} in the context of semi-supervision and object detection. PillarMix takes two points clouds, transforms them with the above-mentioned augmentations (random rotation, flipping and scaling), splits the point clouds into regular square pillars, and mixes them in a checkerboard fashion. It thus ensures that no two (side-)contiguous pillars originate from the same point cloud.
We propose a variant of PillarMix that permits us to achieve better results to train our models (see \cref{sec:ablation}).
For a given reference point cloud, we randomly choose $N-1$ other point clouds, which we set aside. After applying the usual rotate-flip-scale augmentations to each point cloud, we construct a new point cloud by splicing pillars from these $N$ point clouds. Specifically, for each occupied pillar in the reference point cloud, we check whether a pillar \emph{at the same location} is also occupied in \emph{at least one} of the set-aside point clouds. If so, we choose at random one of these occupied pillars, among the up-to $N$ occupied pillars at this location. Otherwise, the pillar of the reference point cloud is kept.
The final point cloud is a mix of points from $N$ point clouds. The truncation of point clouds at pillar edges creates hard samples to learn from.
The mixing creates context variations around objects to segment. Note that, as spliced pillars are from the same location with respect to the ego vehicle, their are consistent regarding the scan pattern (viewpoint, density). There is also a consistency at a more global level as the augmentation replaces an occupied pillar by another occupied pillar, and thus retains the general shape of the reference point cloud.
The choice of number $N$ of point clouds to mix and the size $S$ of pillars is studied in \cref{sec:ablation}. Note that the size $S$ of pillars is much larger (several meters long) than the size $s$ of the pillars used for tokenization (less than one meter).
\pmixplus\ differs from PillarMix in the following ways. With its checkerboard pattern, PillarMix only mixes two point clouds, while our ablations show (\cref{tab:aug_hyperparameters}) that mixing more than two is beneficial. Besides, when mixing point clouds representing different road shapes, thus with entire areas occupied only in one of the two point clouds, the PillarMix checkerboard pattern creates pillars mostly surrounded by empty space (empty pillars from the other point cloud), similar to stand-alone crops, thus reducing the variety of information coming the immediate environment of the pillar. In contrast, as \pmixplus\ preserves pillar occupancy, it can make the most of context variation. Last, the number of occupied pillars in PillarMix can be larger than in the reference point cloud, therefore slightly increasing the number of tokens and the computational load during training. With \pmixplus, the number of occupied pillars stays the same, hence almost preserving the number of tokens.
% ===============================================
%
% ===============================================
\section{Experiments}
\label{sec:}
In this section, we evaluate the performance in semantic segmentation of our proposed VaViT backbones (for sizes S, B, L) across three different lidar datasets. After describing the experimental setup (\cref{sec:exp_setup}), we first benchmark our models against state-of-the-art methods (\cref{sec:exp_main}).
Second, we provide an ablation and sensitivity study to validate our architectural choices for the tokenizer and segmentation head, as well as their core hyperparameters (\cref{sec:ablation}).
We also study the impact of different point cloud augmentation strategies to train our VaViT model, in particular comparing \pmixplus and PolarMix (\cref{sec:plr_mixes}).
Finally, we discuss the robustness of our model to global transformations and the downstream impact when using TTA (\cref{sec:tta}).
\input{tables/main/nuscenes_val_set}
\input{tables/main/waymo_val_set}
\input{tables/main/semkitti_val_set}
% ===============================================
\subsection{Experimental setup}
\label{sec:exp_setup}
\subsubsection{Datasets.}
%
We evaluate our method on three benchmark datasets: nuScenes \cite{nuscenes}, SemanticKITTI \cite{semkitti}, and the Waymo Open Dataset (WOD) \cite{waymo}. The nuScenes dataset consists of 1,000 scenes captured in Boston and Singapore, with point clouds annotated into 16 semantic classes. SemanticKITTI comprises 22 sequences recorded in Karlsruhe, Germany, with 19 semantic classes. Finally, WOD includes scenes from San Francisco, Phoenix and Mountain View, providing semantic masks for 22 distinct classes.
\subsubsection{VaViT backbones.} We experiment with three versions of our backbone: VaViT-S, VaViT-B, VaViT-L, leveraging the ViT-S, ViT-B or ViT-L architecture \cite{dosovitskiy2021image,dino}, respectively. The feature size $F_\ell$ is set to 64/128/192 for the S/B/L versions. This size was set by conducting a sensitivity study on VaViT-S (see \cref{sec:ablation}) and the size was scaled proportionally to the inner dimension of the ViTs. Unless specified otherwise, we use $L_{\rm emb} = 4$ layers for tokenization.
\subsubsection{Baselines.} The closest baselines are PTv3~\cite{ptv3}, LitePT~\cite{litept}, and FlatFormer~\cite{flatformer}. For PTv3, we were able to reproduce the original results on nuScenes and WOD. On SemanticKITTI, reproducing the original is known to be difficult\footnote{https://github.com/Pointcept/Pointcept/issues/186} \cite{dinointheroom}. We were able to reach an mIoU of $68.8\%$ with test time augmentation (TTA) by adapting the configuration provided for nuScenes. This score is in line with the reproduced result in \cite{dinointheroom}. For LitePT, we used the publicly released checkpoints. FlatFormer is originally designed for object detection. We repurposed it for semantic segmentation as follows: we extracted the transformer architecture working in BEV and inserted it between our proposed tokenizer and our semantic segmentation head. We are thus not reusing the tokenizer of FlatFormer, which was in any case designed and tuned for object detection in BEV, not for semantic segmentation. The resulting adaptation of FlatFormer for semantic segmentation is denoted FlatFormer-S.
\subsubsection{Test time augmentation (TTA).} The latest works in point cloud semantic segmentation report scores by resorting to test-time augmentations. It is a practice that has evolved over time. We argue that this setting does not fully show the properties of the backbones. Besides, it is impractical for a number of real-world uses. As a matter of fact, a full evaluation with TTA on the considered validation sets takes several hours for PTv3 and LitePT when using a single GPU. In contrast, it is a matter of minutes for our VaViT models without TTA.
In our main tables, we therefore report results \emph{without} TTA (reproducing results when necessary) and discuss TTA in \cref{sec:tta}.
\subsubsection{Implementation details.} Unless otherwise specified, we train our VaViT models with a batch size of 8, for 45 epochs and a weight decay of $ 0.003$, using AdamW \cite{adamw}. The learning rate schedule starts with a linear warmup during the first 4 epochs to reach $0.001$ and is followed by a cosine decrease to 0. We use standard augmentations: random rotations around the vertical axis, random flip of the horizontal plane, and random scaling in $[0.9, 1.1]$. On SemanticKITTI, we use the CutMix augmentation \cite{RPVNet,2dpass} as available in the official implementation of \cite{waffleiron}. We use stochastic depth \cite{stocdepth} in the ViT backbone with a drop probability of~0.3. For \pmixplus, unless otherwise mentioned, we mix three different point clouds using square pillars of side size 7.5\,m.
\subsection{Comparison to the state of the art}
\label{sec:exp_main}
We report the semantic segmentation performance of various models on the validation sets of nuScenes \cite{nuscenes}, WOD \cite{waymo}, and SemanticKITTI \cite{semkitti} in Tab.~\ref{tab:nuscenes_val_set}, \ref{tab:waymo_val_set}, and \ref{tab:semkitti_val_set}, respectively. While training stochasticity generally has little impact on the final performance with nuScenes, training on WOD and SemanticKITTI may display a visible variance. As the practices differ in the literature regarding citing the performance at last or best epoch, we report for SemanticKITTI and WOD not only VaViT results for our final training epoch but also for our best epoch, denoted VaVit$^*$. The results on SemanticKITTI in this section are obtained with a drop probability of 0.5 instead of 0.3 in stochastic depth \cite{stocdepth}. The size of the pillar for tokenization is set to 0.5\,m for all datasets.
On nuScenes, VaViT-B achieves state-of-the-art performance surpassing the recent LitePT. Interestingly, VaViT-B ranks first on traffic cone and bicycle, despite their small sizes and the coarse grid used for tokenization. On WOD and SemanticKITTI, our models also competes with today's best methods. We notice in particular that VaViT has a strong advantage on truck in SemanticKITTI, probably thanks to the global attention mechanism. Finally, our models rivals well with the other methods on categories of small-size objects, such as bicycle, person/pedestrian, motorcycle, or traffic sign.
% ===============================================
\subsection{Ablation and sensitivity study}
\label{sec:ablation}
In this section, we conduct ablation experiments to validate our architectural designs for our tokenizer and segmentation head, and investigate the sensitivity of VaViT to various hyperparameters. Second, we evaluate the impact of different data augmentation strategies on the final performance. Finally, we present how the performance improves by scaling the capacity of our VaViT backbones.
All experiments in this section are performed using VaViT-S, unless mentionned otherwise. We use a default setting with $L_{\rm emb} = 4$, $F_\ell = 64$, token pillar resolution $1~\text{m}$ for the tokenization, $N = 3$ and $S = 5\,\text{m}$ in \pmixplus.
\input{tables/ablation/architecture2}
\input{tables/ablation/hyperparameters}
\input{tables/ablation/augmentations}
\input{tables/ablation/scale_backbone}
\subsubsection{Tokenizer and segmentation head architectures.}
%
We justify our architectural choices in \cref{tab:architecture}. Starting from a baseline that utilizes mean pooling for BEV projection as used in \cite{waffleiron}, we find that switching to max pooling yields comparable performance; we adopted the latter in the final models. Furthermore, the results demonstrate that incorporating our proposed gating mechanisms into both the tokenizer and the segmentation head provide a consistent performance boost, confirming their relevance. % in refining spatial features.
\subsubsection{Architecture hyperparameters.}
%
We explore the impact of $F_\ell$, $L_{\rm emb}$ and the pillar size for tokenization in \cref{tab:arch_hyperparameters}. \cref{tab:arch_hyperparameters}~(a) shows that the optimal width % \nerminc{it is width, right?}
is $F_\ell = 64$; performance degrades at lower values and plateaus beyond this point. \cref{tab:arch_hyperparameters}~(b) shows that $L_{\rm emb} = 4$ yields the best results. Finally, \cref{tab:arch_hyperparameters}~(c) indicates that segmentation performance improves as the pillar resolution increases (smaller pillar size). We set our resolution limit at $s = 0.5$\,m, as memory consumption begins to scale prohibitively below this threshold.
\subsubsection{\pmixplus and PolarMix augmentations.}
\label{sec:plr_mixes}
%
We now justify our choice of hyperparameters for \pmixplus in \cref{tab:aug_hyperparameters}~(a)-(b). These experiments are done starting from the baseline setting in \cref{tab:architecture}, i.e., with mean pooling and without the gating mechanisms. We notice that the best results are obtained by mixing pillars of size $S = 7.5~\text{m}$ and considering $N = 3$ point clouds for mixing. The last row in \cref{tab:aug_hyperparameters}~(b) shows that our improved version \pmixplus performs better than the PillarMix version proposed in \cite{pillarmix}.
We also compare \pmixplus against PolarMix \cite{polarmix} as a potential alternative in \cref{tab:aug_hyperparameters} (c). In our implementation, PolarMix partitions the scene into $N$ random angular sectors for mixing. While mixing three point clouds also yields the best performance for the PolarMix variant, its overall scores remain lower than those achieved with \pmixplus.
\subsubsection{Capacity of VaViT models.}
\label{sec:capacity}
We experiments with different model capacity in \cref{tab:capacity}. The VaViT-S, VaViT-B and VaViT-L models have respectively $21$M, $86$M and $304$M parameters. The results shows that increasing the model capacity from VaViT-S to VaViT-B improves the performance, except on SemanticKITTI where its stagnates. This is probably partly explained by the fact that SemanticKITTI is the smallest dataset among all three. Increasing the capacity further up to VaViT-L does not improve performance. We hypothesize that the scale of the datasets is not large enough to witness improved performance with VaViT-L, or that even stronger augmentations must be used.
% ===============================================
\subsection{Robustness to global transformations and TTA}
\label{sec:tta}
\input{tables/main/tta}
A classical technique to improve performance is to use TTA. In practice, a point cloud is transformed using global transformation such as rotation around the vertical axis, scaling, and flip of the horizontal plane. While, ideally, one would like a trained network to be robust to these transformations (used during the training phase), most networks are not. Ensembling the predictions over multiple such transformations then boost the performance. This technique permits to gain about $2$ mIoU points for PTv3 and LitePT (see the first two rows of \cref{tab:tta}).
In contrast, VaViT models are much more robust to such global transformation. The results in the last row of \cref{tab:tta} demonstrates this behavior as the scores are nearly unchanged after applying TTA\footnote{We used 12 different random augmentations.} with global transformations: the transformations have been mostly learned by the networks.
We explain this behavior by the use of global self-attentions in the ViT, instead of local attentions as in PTv3 and LitePT. From a global point of view, the structure of the point cloud is unchanged after applying global transformations, which the global attention mechanism is more likely to be robust to, as it has a view on the entire point cloud. The downside is that TTA with such transformations is not a technique that significantly boost performance of our VaViT models.
% ===============================================
%
% ===============================================
\section{Conclusion}
We have shown that it is possible to leverage a plain ViT architecture and compete with state-of-the-art methods for semantic segmentation of automotive lidar point clouds. The main ingredients to achieve this result are the careful design of both the tokenizer and the segmentation head, as well as sufficiently strong augmentations.
We believe this work can open several doors, including easier multimodal architecture designs, mixing point clouds and images, and the direct application of self-supervised pretraining techniques for transformers, which have proven powerful in other domains.
% ===============================================
%
% ===============================================
{
\bibliographystyle{ieeenat_fullname}
\bibliography{main} % .bib
}
\newpage
\appendix
\section{Appendix}
\input{appendix/supp}
\end{document}
\begin{table*}[t]
\caption{\textbf{Performance on nuScenes \emph{validation} set.} We report results \emph{without} test-time augmentation. Best and second-best mIoUs are indicated by \textbf{bold} and \underline{underlining}. $^\dagger$~indicates results we reproduced.}
\label{tab:nuscenes_val_set}
\small
\ra{1.3}
\newcommand*\rotext{\multicolumn{1}{R{65}{1em}}}
\centering
\setlength{\tabcolsep}{3pt}
\begin{tabular}{l@{}r c | c c c c c c c c c c c c c c c c}
\toprule
Method
&
& \rotext{\bf mIoU\%}
& \rotext{barrier}
& \rotext{bicycle}
& \rotext{bus}
& \rotext{car}
& \rotext{const. veh.}
& \rotext{motorcycle}
& \rotext{pedestrian}
& \rotext{traffic cone}
& \rotext{trailer}
& \rotext{truck}
& \rotext{driv. surf.}
& \rotext{other flat}
& \rotext{sidewalk}
& \rotext{terrain}
& \rotext{manmade}
& \rotext{vegetation}
\\
\midrule
GFNet
& \cite{gfnet}
& 76.1
& 81.1
& 31.6
& 76.0
& 90.5
& 60.2
& 80.7
& 75.3
& 71.8
& 82.5
& 65.1
& 97.8
& 67.0
& 80.4
& 76.2
& 91.8
& 88.9
\\
Cylinder3D
& \cite{Cylinder3d}
& 76.1
& 76.4
& 40.3
& 91.2
& 93.8
& 51.3
& 78.0
& 78.9
& 64.9
& 62.1
& 84.4
& 96.8
& 71.6
& 76.4
& 75.4
& 90.5
& 87.4
\\
RPVNet
& \cite{RPVNet}
& 77.6
& 78.2
& 43.4
& 92.7
& 93.2
& 49.0
& 85.7
& 80.5
& 66.0
& 66.9
& 84.0
& 96.9
& 73.5
& 75.9
& 76.0
& 90.6
& 88.9
\\
WI-384
& \cite{waffleiron}
& 77.6
& 78.7
& 51.3
& 93.6
& 88.2
& 47.2
& 86.5
& 81.7
& 68.9
& 69.3
& 83.1
& 96.9
& 74.3
& 75.6
& 74.2
& 87.2
& 85.2
\\
SDSeg3D
& \cite{sdseg3d}
& 77.7
& 77.5
& 49.4
& 93.9
& 92.5
& 54.9
& 86.7
& 80.1
& 67.8
& 65.7
& 86.0
& 96.4
& 74.0
& 74.9
& 74.5
& 86.0
& 82.8
\\
RangeFormer
& \cite{rangeformer}
& 78.1
& 78.0
& 45.2
& 94.0
& 92.9
& 58.7
& 83.9
& 77.9
& 69.1
& 63.7
& 85.6
& 96.7
& 74.5
& 75.1
& 75.3
& 89.1
& 87.5
\\
SphereFormer
& \cite{sphereformer}
& 78.4
& 77.7
& 43.8
& 94.5
& 93.1
& 52.4
& 86.9
& 81.2
& 65.4
& 73.4
& 85.3
& 97.0
& 73.4
& 75.4
& 75.0
& 91.0
& 89.2
\\
FlatFormer-S$^\dagger$
& \cite{flatformer}
& 78.6
& 78.8
& 45.4
& 94.5
& 93.0
& 58.1
& 81.5
& 79.9
& 67.2
& 71.5
& 85.2
& 96.8
& 74.5
& 75.9
& 75.4
& 90.8
& 89.2
\\
WI-768
& \cite{scalr}
& 78.7
& 79.2
& 53.2
& 92.5
& 88.1
& 50.4
& 87.8
& 83.4
& 70.3
& 73.5
& 84.1
& 96.9
& 73.3
& 75.5
& 75.4
& 89.9
& 86.5
\\
PTv3$^\dagger$
& \cite{ptv3}
& 78.9
& 79.7
& 49.7
& 95.6
& 93.6
& 46.0
& 84.6
& 82.2
& 70.1
& 72.9
& 84.0
& 96.9
& 74.7
& 76.3
& 75.5
& 90.8
& 89.1
\\
LitePT$^\dagger$
& \cite{litept}
& \underline{80.7}
& 79.2
& 52.3
& 95.7
& 94.2
& 64.4
& 84.8
& 82.8
& 68.9
& 72.9
& 87.7
& 97.0
& 76.7
& 77.3
& 76.8
& 91.1
& 89.6
\\
\rowcolor{blue!15}
VaViT-B
& \llap{(ours)}
& \bf 81.3
& 79.6
& 54.4
& 95.2
& 93.9
& 65.4
& 88.8
& 83.6
& 72.0
& 74.1
& 87.3
& 97.1
& 75.3
& 77.0
& 76.2
& 91.3
& 89.7
\\
\bottomrule
\end{tabular}
\end{table*}
\begin{table*}[t]
\caption{\textbf{Performance on the \emph{validation} split of the WOD.} We report results \emph{without} test-time augmentation. Best and second-best mIoUs are indicated by \textbf{bold} and \underline{underlining}.
%
Because of space constraint, we removed the easy classes `car' and `building' with IoUs above $94\%$ for all methods. $^\dagger$ indicates results we reproduced.}
\label{tab:waymo_val_set}
\small
\ra{1.3}
\newcommand*\rotext{\multicolumn{1}{R{65}{1em}}}
% \newcommand*\rotext{{}}
\centering
\setlength{\tabcolsep}{2pt}
\begin{tabular}{l c | c c c c c c c c c c c c c c c c c c c c}
\toprule
Method
& \rotext{\bf mIoU\%}
% & \rotext{car}
& \rotext{truck}
& \rotext{bus}
& \rotext{oth.~veh.}
& \rotext{motorcyclist}
& \rotext{bicyclist}
& \rotext{pedestrian}
& \rotext{sign}
& \rotext{traffic light}
& \rotext{pole}
& \rotext{const.~cone}
& \rotext{bicycle}
& \rotext{motorcycle}
% & \rotext{building}
& \rotext{vegetation}
& \rotext{trunk}
& \rotext{curb}
& \rotext{road}
& \rotext{lane marker}
& \rotext{other ground}
& \rotext{walkable}
& \rotext{sidewalk}
\\
\midrule
SphereFormer
& 69.9
% & 94.5
& 61.6
& 87.7
& 40.2
& \phantom{0}0.9
& 69.7
& 90.2
& 73.9
& 41.8
& 77.2
& 65.4
& 71.9
& 83.7
% & 95.9
& 91.7
& 68.4
& 69.8
& 93.3
& 53.9
& 47.9
& 80.8
& 77.2
\\
PTv3$^\dagger$
& 70.2
% & 94.8
& 63.3
& 83.9
& 34.6
& \phantom{0}3.9
& 77.1
& 90.4
& 70.8
& 32.7
& 77.3
& 67.1
& 74.3
& 88.0
% & 95.8
& 91.7
& 67.5
& 70.8
& 93.4
& 56.6
& 52.0
& 81.5
& 77.8
\\
\rowcolor{blue!15}VaViT-B
& 70.5
% & 94.4
& 61.3
& 84.2
& 42.8
& 10.2
& 74.7
& 90.8
& 74.4
& 33.7
& 79.5
& 63.3
& 73.8
& 80.6
% & 96.1
& 92.0
& 69.9
& 71.1
& 93.5
& 55.4
& 50.3
& 81.4
& 78.3
\\
\rowcolor{blue!15}VaViT-B*
& \underline{70.9}
% & 94.5
& 62.5
& 86.7
& 39.3
& \phantom{0}7.7
& 76.9
& 90.8
& 74.0
& 33.6
& 79.6
& 66.6
& 74.4
& 87.2
% & 96.0
& 91.9
& 70.2
& 70.8
& 93.4
& 55.1
& 49.9
& 81.4
& 78.3
\\
LitePT$^\dagger$
& \bf 71.7
% & 94.6
& 63.1
& 83.9
& 41.2
& \phantom{0}9.8
& 78.4
& 90.7
& 75.0
& 39.5
& 79.0
& 74.5
& 73.6
& 87.6
% & 95.9
& 91.8
& 70.0
& 69.8
& 93.2
& 56.3
& 51.1
& 81.0
& 77.3
\\
\bottomrule
\end{tabular}
\end{table*}
\begin{table*}[t]
\caption{\textbf{Performance on SemanticKITTI \emph{validation} set.} We report results \emph{without} test-time augmentation. Best and second-best mIoUs are indicated by \textbf{bold} and \underline{underlining}.
%
$^\dagger$ indicates results we reproduced. The mIoU for MinkUNet, SPVNAS and Cylinder 3D and SphereFormer are obtained from \cite{ptv3}.}
\label{tab:semkitti_val_set}
\small
\newcommand*\rotext{\multicolumn{1}{R{65}{1em}}}
\centering
\ra{1.3}
\setlength{\tabcolsep}{2pt}
\begin{tabular}{l c | c c c c c c c c c c c c c c c c c c c}
\toprule
Method
& \rotext{\bf mIoU\%}
& \rotext{car}
& \rotext{bicycle}
& \rotext{motorcycle}
& \rotext{truck}
& \rotext{other-vehicle}
& \rotext{person}
& \rotext{bicyclist}
& \rotext{motorcyclist}
& \rotext{road}
& \rotext{parking}
& \rotext{sidewalk}
& \rotext{other-ground}
& \rotext{building}
& \rotext{fence}
& \rotext{vegetation}
& \rotext{trunk}
& \rotext{terrain}
& \rotext{pole}
& \rotext{traffic-sign}
\\
\midrule
MinkUnet
& 63.8
&-&-&-&-&-&-&-&-&-&-&-&-&-&-&-&-&-&-&-
\\
SPVNAS
& 64.7
&-&-&-&-&-&-&-&-&-&-&-&-&-&-&-&-&-&-&-
\\
Cylinder3D
& 64.3
&-&-&-&-&-&-&-&-&-&-&-&-&-&-&-&-&-&-&-
\\
FlatFormer-S$^\dagger$ % lr 0.0002
& 65.3
& 94.9
& 56.8
& 75.8
& 88.0
& 40.7
& 77.3
& 91.6
& 0.0
& 94.5
& 44.9
& 82.7
& 0.2
& 90.6
& 60.7
& 87.2
& 69.5
& 72.0
& 64.8
& 48.5
\\
PTv3$^\dagger$
& 66.2
& 96.2
& 45.2
& 79.0
& 81.9
& 55.6
& 79.0
& 91.0
& 0.0
& 95.3
& 53.2
& 84.4
& 00.3
& 90.3
& 60.4
& 87.6
& 70.3
& 73.0
& 66.8
& 48.2
\\
\rowcolor{blue!15}VaViT-B
& 67.6
& 95.7
& 56.5
& 74.9
& 95.4
& 52.1
& 81.4
& 94.5
& 0.1
& 95.6
& 51.1
& 83.8
& 0.1
& 90.6
& 61.0
& 88.1
& 72.0
& 73.5
& 65.9
& 51.6
\\
SphereFormer
& \underline{67.8}
&-&-&-&-&-&-&-&-&-&-&-&-&-&-&-&-&-&-&-
\\
\rowcolor{blue!15}VaViT-B*
& \bf 68.0
& 96.3
& 53.2
& 76.3
& 95.2
& 62.1
& 80.1
& 94.0
& 0.0
& 95.8
& 51.9
& 83.9
& 0.2
& 90.7
& 62.9
& 87.9
& 71.9
& 72.9
& 65.7
& 50.7
\\
WI-256
& \bf 68.0
& 96.1
& 58.1
& 79.7
& 77.4
& 59.0
& 81.1
& 92.2
& 1.3
& 95.5
& 50.2
& 83.6
& 6.0
& 92.1
& 67.5
& 87.8
& 73.8
& 73.0
& 65.7
& 52.2
\\
\bottomrule
\end{tabular}
\end{table*}
%
\begin{table}[t]
%
\caption{\textbf{Ablation of the tokenizer and segmentation head components.} We compare different pooling strategies for BEV projection and evaluate the impact of the proposed gating mechanisms. Results are reported in mIoU (\%) on the validation sets of SemanticKITTI and WOD. Incorporating gating in both the tokenizer and the segmentation head consistently improves the mIoU over the baseline.}
\label{tab:architecture}
\centering
\small
\ra{1.2}
\setlength{\tabcolsep}{6pt}
\begin{tabular}{cccccc}
\toprule
Mean
& Max
& Token gat.
& Seg.\,gat.
&
&
\\[-0.5mm]
pool
& pool
& $f_{\rm gate}(\cdot)$
& $g_{\rm gate}(\cdot)$
& \!SemKITTI\!
& WOD
\\
\cmidrule(lr){1-4}
\cmidrule(lr){5-6}
\cmark
& -
& \xmark
& \xmark
& 63.5
& 66.9
\\
-
& \cmark
& \xmark
& \xmark
& 63.6
& 66.9
\\
-
& \cmark
& \cmark
& \xmark
& 65.3
& 68.9
\\
\rowcolor{blue!15}-
& \cmark
& \cmark
& \cmark
& \bf 66.0
& \bf 69.2
\\
\bottomrule
\end{tabular}
\end{table}
%
%
\begin{table}[t]
\centering
\caption{\textbf{Sensitivity analysis of architecture hyperparameters.} We evaluate the impact of (a) the feature dimension $F_\ell$ , (b) the number $L_{\rm emb}$ of layers in the tokenizer, and (c) the pillar resolution $s$ for the tokenizer.
}
\label{tab:arch_hyperparameters}
\begin{minipage}[t]{0.31\linewidth}
\centering
\scriptsize
\ra{1.1}
\setlength{\tabcolsep}{3pt}
\begin{tabular}[t]{ccc}
\toprule
$F_\ell$
& SemKITTI
& WOD
\\
\cmidrule(lr){1-1}
\cmidrule(lr){2-3}
\phantom{1}16
& 63.6
& 66.3
\\
\phantom{1}32
& 64.8
& 67.9
\\
\rowcolor{blue!15}\phantom{1}64
& 66.0
& \bf 69.2
\\
\phantom{1}96
& 66.0
& 68.5
\\
128
& \bf 66.1
& 68.9
\\
\bottomrule
\end{tabular}
\vspace{1mm}
\\
(a) $L_{\rm emb}=4$; $s=1$\,m.
\end{minipage}
\hspace*{-7.5pt}\hfill
%
%
%
\begin{minipage}[t]{0.31\linewidth}
\centering
\scriptsize
\ra{1.1}
\setlength{\tabcolsep}{3pt}
\begin{tabular}[t]{ccc}
\toprule
$L_{\rm emb}$
& SemKITTI
& WOD
\\
\cmidrule(lr){1-1}
\cmidrule(lr){2-3}
1
& 64.2
& 67.2
\\
2
& 64.4
& 68.3
\\
3
& 65.6
& 68.3
\\
\rowcolor{blue!15} 4
& \bf 66.0
& \bf 69.2
\\
5
& 65.2
& 68.3
\\
\bottomrule
\end{tabular}
\vspace{1mm}
\\
(b) $F_\ell = 64$; $s=1$\,m.
\end{minipage}
\hfill
%
%
%
\begin{minipage}[t]{0.31\linewidth}
\centering
\scriptsize
\ra{1.1}
\setlength{\tabcolsep}{3pt}
\begin{tabular}[t]{ccc}
\toprule
$s$ (m)
& SemKITTI
& WOD
\\
\cmidrule(lr){1-1}
\cmidrule(lr){2-3}
\rowcolor{blue!15} 0.50
& \bf 67.1
& \bf 69.4
\\
0.75
& 66.6
& 69.3
\\
1.00
& 66.0
& 69.2
\\
1.25
& 64.8
& 68.4
\\
\bottomrule
\end{tabular}
\vspace{1mm}
\\
(c) $L_{\rm emb}=4$; $F_\ell = 64$.
\end{minipage}
\end{table}
%
%
\begin{table}[t]
\caption{\textbf{Hyperparameter study for \pmixplus and PolarMix.} We evaluate the hyperparameters for \pmixplus in (a) and (b), finding that mixing three point clouds with a pillar size $S$ of 7.5 m yields the best performance. In~(c), we compare \pmixplus against PolarMix \cite{polarmix}. While both benefit from mixing $N=3$ samples, \pmixplus outperforms PolarMix. Finally, (b)~also shows that \pmixplus outperforms PillarMix \cite{pillarmix}, justifying our improvement.}
\label{tab:aug_hyperparameters}
\centering
%
%
%
\begin{minipage}[t]{0.31\linewidth}
\centering
\scriptsize
\ra{1.1}
\setlength{\tabcolsep}{2pt}
\begin{tabular}[t]{ccc}
\toprule
$S$ (m)
& SemKITTI
& WOD
\\
\cmidrule(lr){1-1}
\cmidrule(lr){2-3}
\phantom{0}0.0
& 62.3
& 64.9
\\
\phantom{0}2.5
& 63.2
& 66.3
\\
\phantom{0}5.0
& 63.5
& 66.9
\\
\rowcolor{blue!15}\phantom{0}7.5
& \bf 63.8
& \bf 67.1
\\
10.0
& 63.2
& 66.5
\\
\bottomrule
\end{tabular}
\vspace{1mm}
\\
(a) \pmixplus - $N = 3$
\end{minipage}
\hspace*{10pt}\hfill
%
%
%
\begin{minipage}[t]{0.31\linewidth}
\centering
\scriptsize
\ra{1.1}
\setlength{\tabcolsep}{2pt}
\begin{tabular}[t]{ccc}
\toprule
$N$
& SemKITTI
& WOD
\\
\cmidrule(lr){1-1}
\cmidrule(lr){2-3}
2
& 63.7
& 66.4
\\
\rowcolor{blue!15}3
& \bf 63.8
& \bf 67.1
\\
4
& 63.8
& 66.9
\\
5
& 63.4
& 67.1
\\
\midrule
\cite{pillarmix}
& 63.7
& 66.3
\\
\bottomrule
\end{tabular}
\vspace{1mm}
\\
(b) \pmixplus - $S = 7.5 \text{m}$
\end{minipage}
\hfill
%
%
%
%
\begin{minipage}[t]{0.31\linewidth}
\centering
\scriptsize
\ra{1.1}
\setlength{\tabcolsep}{2pt}
\begin{tabular}[t]{ccc}
\toprule
$N$
& SemKITTI
& WOD
\\
\cmidrule(lr){1-1}
\cmidrule(lr){2-3}
2 & 62.9 & 66.0 \\
3 & \bf 63.3 & \bf 66.0 \\
4 & 62.5 & 65.6 \\
5 & 62.6 & 66.1 \\
\bottomrule
\end{tabular}
\vspace{1mm}
\\
(c) PolarMix
\end{minipage}
\end{table}
%
\begin{table}[t]
\caption{\textbf{Effect of model capacity.} We report the mIoU obtained on the validation sets of nuScenes, SemanticKITTI and WOD using a VaViT-S, VaViT-B and VaViT-L models. The best performance is obtained with VaViT-B.}
\label{tab:capacity}
\small
\centering
\ra{1.1}
\setlength{\tabcolsep}{8pt}
\begin{tabular}{lccc}
\toprule
Method
& nuScenes
& SemKITTI
& WOD
\\
\midrule
VaViT-S
& 80.6
& \bf 67.6
& 70.1
\\
\rowcolor{blue!15}VaViT-B
& \bf 81.3
& \bf 67.6
& \bf 70.5
\\
VaViT-L
& 81.0
& -
& 70.2
\\
\bottomrule
\end{tabular}
\end{table}
\begin{table}[t]
\caption{\textbf{Robustness to global transformations.} We report the mIoU\% obtained on the validation sets of nuScenes, SemanticKITTI and WOD for PTV3, LitePT and VaViT-B* with and without TTA. $^\dagger$ indicates results we reproduced.}
\label{tab:tta}
\scriptsize
\centering
\ra{1.2}
\setlength{\tabcolsep}{4.5pt}
\begin{tabular}{lcccccc}
\toprule
\multirow{2}{*}{Method}
& \multicolumn{2}{c}{nuScenes}
& \multicolumn{2}{c}{SemKITTI}
& \multicolumn{2}{c}{WOD}
\\
\cmidrule(lr){2-3}
\cmidrule(lr){4-5}
\cmidrule(lr){6-7}
& w/o TTA
& w/~TTA
& w/o TTA
& w/~TTA
& w/o TTA
& w/~TTA
\\
\midrule
PTv3$^\dagger$
& 78.9
& 80.4
& 66.2
& \bf 68.8
& 70.2
& 71.7
\\
LitePT$^\dagger$
& 80.7
& \bf 82.2
& -
& -
& \bf 71.7
& \bf 73.2
\\
\rowcolor{blue!15}VaViT-B*
& \bf 81.3
& 81.6
& \bf 68.0
& 68.7
& 70.9
& 71.4
\\
\bottomrule
\end{tabular}
\end{table}
% \documentclass[runningheads]{llncs}
% % ---------------------------------------------------------------
% % Include basic ECCV package
% % TODO REVIEW: Insert your submission number below by replacing '*****'
% % TODO FINAL: Comment out the following line for the camera-ready version
% \usepackage[review,year=2026,ID=9564]{eccv}
% % TODO FINAL: Un-comment the following line for the camera-ready version
% %\usepackage{eccv}
% % ---------------------------------------------------------------
% % Other packages
% % Commonly used abbreviations (\eg, \ie, \etc, \cf, \etal, etc.)
% \usepackage{eccvabbrv}
% % Include other packages here, before hyperref.
% \usepackage{pifont}
% \usepackage{graphicx}
% \usepackage{booktabs}
% \usepackage{multirow}
% \usepackage{adjustbox}
% \usepackage{array}
% \newcolumntype{R}[2]{%
% >{\adjustbox{angle=#1,lap=1.3\width-(#2)}\bgroup}%
% l%
% <{\egroup}%
% }
% \usepackage[table, dvipsnames]{xcolor}
% \newcommand{\ra}[1]{\renewcommand{\arraystretch}{#1}}
% % The "axessiblity" package can be found at: https://ctan.org/pkg/axessibility?lang=en
% \usepackage[accsupp]{axessibility} % Improves PDF readability for those with disabilities.
% % ---------------------------------------------------------------
% % Hyperref package
% % It is strongly recommended to use hyperref, especially for the review version.
% % Please disable hyperref *only* if you encounter grave issues.
% % hyperref with option pagebackref eases the reviewers' job, but should be disabled for the final version.
% %
% % If you comment hyperref and then uncomment it, you should delete
% % main.aux before re-running LaTeX.
% % (Or just hit 'q' on the first LaTeX run, let it finish, and you
% % should be clear).
% % TODO FINAL: Comment out the following line for the camera-ready version
% %\usepackage[pagebackref,breaklinks,colorlinks,citecolor=eccvblue]{hyperref}
% % TODO FINAL: Un-comment the following line for the camera-ready version
% \usepackage[pagebackref]{hyperref}
% % Support for ORCID icon
% \usepackage{orcidlink}
% \definecolor{turquoise}{rgb}{0.20, 0.70, 0.65}
% \def\pmixplus{PillarMix\textsuperscript{\smash+}\xspace}
% \definecolor{better}{rgb}{0.19, 0.55, 0.91}
% \definecolor{worse}{rgb}{0.82, 0.1, 0.26}
% \newcommand{\cmark}{\textcolor{better}{\ding{51}}}
% \newcommand{\xmark}{\textcolor{worse}{\ding{55}}}
% \begin{document}
% % TODO REVIEW: Replace with your title
% \title{Vanilla ViT for Automotive Point Cloud Semantic Segmentation\\--- \textit{Supplementary material} ---}
% % TODO REVIEW: If the paper title is too long for the running head, you can set
% % an abbreviated paper title here. If not, comment out.
% \titlerunning{VaViT}
% \maketitle
\appendix
\setcounter{figure}{1}
\setcounter{table}{8}
% ======================================
%
% ======================================
\section{Details about our adaptation of FlatFormer}
\label{sec:flatformer}
FlatFormer \cite{flatformer} was designed for object detection using a BEV representation of point clouds.
It uses a flattened window attention mechanism and alternates flattening directions within groups.
As mentioned in the main paper, we adapt it to semantic segmentation for comparison purpose and refer to it as Flatformer-S.
We directly replace our ViT by Flatformer's transformer architecture.
That is, the proposed Flatformer-S uses our tokenizer and our segmentation head.
%
In our implementation of Flatformer-S, we adapt the dimensions and number of blocks to match the ViT-S architecture.
We use a feature dimension of 384, 3 groups of 4 flattened attention blocks (12 transformer blocks) and 6 heads.
The BEV resolution is the same as for VaViT for a fair comparison.
%
We report in \cref{tab:flatformer-s} the scores from the main paper corresponding to the same transformer backbone scale (ViT-S).
\begin{table}[h]
\vspace{-1em}
\caption{Semantic segmentation performance (mIoU\%) of Flatformer-S and VaViT-S %.}} %
on the validation sets of nuScenes and SemanticKITTI}.
\label{tab:flatformer-s}
\small
\ra{1.2}
\newcommand*\rotext{\multicolumn{1}{R{65}{1em}}}
\centering
\setlength{\tabcolsep}{4pt}
\begin{tabular}{l | c c}
\toprule
Method
& nuScenes
& SemanticKITTI
\\
\midrule
Flatformer-S & 78.6 & 65.3\\
VaViT-S & \bf 80.6 & \bf 67.6\\
\bottomrule
\end{tabular}
\vspace{-2em}
\end{table}
% ======================================
%
% ======================================
\section{Attention maps}
\label{sec:attention}
We present attention maps at the first, middle and last layers of our VaViT-B models when trained on
nuScenes \cite{nuscenes} in \cref{fig:attention_nuscenes_sidewalk_first,fig:attention_nuscenes_sidewalk_mid,fig:attention_nuscenes_sidewalk_last,fig:attention_nuscenes_car_first,fig:attention_nuscenes_car_mid,fig:attention_nuscenes_car_last},
SemanticKITTI \cite{semkitti} in
\cref{fig:attention_kitti_sidewalk_first,fig:attention_kitti_sidewalk_mid,fig:attention_kitti_sidewalk_last,fig:attention_kitti_car_first,fig:attention_kitti_car_mid,fig:attention_kitti_car_last},
and WOD \cite{waymo} in
\cref{fig:attention_waymo_car_first,fig:attention_waymo_car_mid,fig:attention_waymo_car_last,fig:attention_waymo_pedestrian_first,fig:attention_waymo_pedestrian_mid,fig:attention_waymo_pedestrian_last}. The attention maps are presented at Layer 1, 6 and 12 of the VaViT-B backbones. We use query points on car and sidewalk for nuScenes and SemanticKITTI, and car and pedestrian on WOD.
In general, as we go deeper in the network, the attention tends to concentrate on regions containing points of similar semantics.
It is interesting to see, e.g., that
when querying a point on a pedestrian, the attention maps in Head~5 of Layer~6 (\cref{fig:attention_waymo_pedestrian_mid}) and Head~4 of Layer~12 highlight the other pedestrians in the scene (\cref{fig:attention_waymo_pedestrian_last}).
% ======================================
%
% ======================================
\section{Additional classwise result for WOD}
We present in \cref{tab:waymo_val_set_car_building} the missing classwise results (because of space constraints) for the classes `car' and `building' in Tab.~2.
\begin{table}[t]
\caption{Performance on the \emph{validation} split of WOD \cite{waymo} for the classes `car' and `building'. $^\dagger$ indicates results we reproduced.}
\label{tab:waymo_val_set_car_building}
\small
\ra{1.3}
\newcommand*\rotext{\multicolumn{1}{R{65}{1em}}}
\centering
\setlength{\tabcolsep}{4pt}
\begin{tabular}{l | c c}
\toprule
Method
& \rotext{car}
& \rotext{building}
\\
\midrule
SphereFormer
& 94.5
& 95.9
\\
PTv3$^\dagger$
& 94.8
& 95.8
\\
\rowcolor{blue!15}VaViT-B
& 94.4
& 96.1
\\
\rowcolor{blue!15}VaViT-B*
& 94.5
& 96.0
\\
LitePT$^\dagger$
& 94.6
& 95.9
\\
\bottomrule
\end{tabular}
\end{table}
% ======================================
%
% ======================================
\section{Inference time}
\label{sec:inference_time}
On average over the validation set of nuScenes, the inference time per point cloud for LitePT \cite{litept} is 295 ms with chunking and 42 ms without chunking. We used the official test script to get these timings. For VaViT-B, the inference time per point cloud is 20 ms only. Note that, for all methods, the preprocessing in the dataloader was not measured and TTA was disabled. The experiments were conducted on the same NVIDIA H100.
% ======================================
%
% ======================================
\begin{figure*}
\centering
\includegraphics[width=.85\linewidth]{images/attentions/nuscenes/nuscenes_2200-class_12-layer_0_compressed.pdf}
\caption{\textbf{Attention maps} for each head at the \textbf{1$^{\rm st}$} (first) layer of our VaViT-B model trained on \textbf{nuScenes}. The query point, denoted by a red cross, is located on the \textbf{sidewalk}. Ground truth in BEV is presented at the top. Subsequent maps have a transparency scaled by the attention weight between the query point and the keys; points with zero attention are fully transparent.}
\label{fig:attention_nuscenes_sidewalk_first}
\end{figure*}
\begin{figure*}
\centering
\includegraphics[width=.85\linewidth]{images/attentions/nuscenes/nuscenes_2200-class_12-layer_5_compressed.pdf}
\caption{\textbf{Attention maps} for each head at the \textbf{6$^{\rm th}$} (middle) layer of our VaViT-B model trained on \textbf{nuScenes}. The query point, denoted by a red cross, is located on the \textbf{sidewalk}. Ground truth in BEV is presented at the top. Subsequent maps have a transparency scaled by the attention weight between the query point and the keys; points with zero attention are fully transparent.}
\label{fig:attention_nuscenes_sidewalk_mid}
\end{figure*}
\begin{figure*}
\centering
\includegraphics[width=.85\linewidth]{images/attentions/nuscenes/nuscenes_2200-class_12-layer_11_compressed.pdf}
\caption{\textbf{Attention maps} for each head at the \textbf{12$^{\rm th}$} (final) layer of our VaViT-B model trained on \textbf{nuScenes}. The query point, denoted by a red cross, is located on the \textbf{sidewalk}. Ground truth in BEV is presented at the top. Subsequent maps have a transparency scaled by the attention weight between the query point and the keys; points with zero attention are fully transparent.}
\label{fig:attention_nuscenes_sidewalk_last}
\end{figure*}
\begin{figure*}
\centering
\includegraphics[width=.85\linewidth]{images/attentions/nuscenes/nuscenes_2200-class_3-layer_0_compressed.pdf}
\caption{\textbf{Attention maps} for each head at the \textbf{1$^{\rm st}$} (first) layer of our VaViT-B model trained on \textbf{nuScenes}. The query point, denoted by a red cross, is located on a \textbf{car}. Ground truth in BEV is presented at the top. Subsequent maps have a transparency scaled by the attention weight between the query point and the keys; points with zero attention are fully transparent.}
\label{fig:attention_nuscenes_car_first}
\end{figure*}
\begin{figure*}
\centering
\includegraphics[width=.85\linewidth]{images/attentions/nuscenes/nuscenes_2200-class_3-layer_5_compressed.pdf}
\caption{\textbf{Attention maps} for each head at the \textbf{6$^{\rm th}$} (middle) layer of our VaViT-B model trained on \textbf{nuScenes}. The query point, denoted by a red cross, is located on a \textbf{car}. Ground truth in BEV is presented at the top. Subsequent maps have a transparency scaled by the attention weight between the query point and the keys; points with zero attention are fully transparent.}
\label{fig:attention_nuscenes_car_mid}
\end{figure*}
\begin{figure*}
\centering
\includegraphics[width=.85\linewidth]{images/attentions/nuscenes/nuscenes_2200-class_3-layer_11_compressed.pdf}
\caption{\textbf{Attention maps} for each head at the \textbf{12$^{\rm th}$} (last) layer of our VaViT-B model trained on \textbf{nuScenes}. The query point, denoted by a red cross, is located on a \textbf{car}. Ground truth in BEV is presented at the top. Subsequent maps have a transparency scaled by the attention weight between the query point and the keys; points with zero attention are fully transparent.}
\label{fig:attention_nuscenes_car_last}
\end{figure*}
\begin{figure*}
\centering
\includegraphics[width=.85\linewidth]{images/attentions/kitti/kitti_2200-class_10-layer_0_compressed.pdf}
\caption{\textbf{Attention maps} for each head at the \textbf{1$^{\rm st}$} (first) layer of our VaViT-B model trained on \textbf{SemanticKITTI}. The query point, denoted by a red cross, is located on the \textbf{sidewalk}. Ground truth in BEV is presented at the top. Subsequent maps have a transparency scaled by the attention weight between the query point and the keys; points with zero attention are fully transparent.}
\label{fig:attention_kitti_sidewalk_first}
\end{figure*}
\begin{figure*}
\centering
\includegraphics[width=.85\linewidth]{images/attentions/kitti/kitti_2200-class_10-layer_5_compressed.pdf}
\caption{\textbf{Attention maps} for each head at the \textbf{6$^{\rm th}$} (middle) layer of our VaViT-B model trained on \textbf{SemanticKITTI}. The query point, denoted by a red cross, is located on the \textbf{sidewalk}. Ground truth in BEV is presented at the top. Subsequent maps have a transparency scaled by the attention weight between the query point and the keys; points with zero attention are fully transparent.}
\label{fig:attention_kitti_sidewalk_mid}
\end{figure*}
\begin{figure*}
\centering
\includegraphics[width=.85\linewidth]{images/attentions/kitti/kitti_2200-class_10-layer_11_compressed.pdf}
\caption{\textbf{Attention maps} for each head at the \textbf{12$^{\rm th}$} (last) layer of our VaViT-B model trained on \textbf{SemanticKITTI}. The query point, denoted by a red cross, is located on the \textbf{sidewalk}. Ground truth in BEV is presented at the top. Subsequent maps have a transparency scaled by the attention weight between the query point and the keys; points with zero attention are fully transparent.}
\label{fig:attention_kitti_sidewalk_last}
\end{figure*}
\begin{figure*}
\centering
\includegraphics[width=.85\linewidth]{images/attentions/kitti/kitti_2200-class_0-layer_0_compressed.pdf}
\caption{\textbf{Attention maps} for each head at the \textbf{1$^{\rm st}$} (first) layer of our VaViT-B model trained on \textbf{SemanticKITTI}. The query point, denoted by a red cross, is located on a \textbf{car}. Ground truth in BEV is presented at the top. Subsequent maps have a transparency scaled by the attention weight between the query point and the keys; points with zero attention are fully transparent.}
\label{fig:attention_kitti_car_first}
\end{figure*}
\begin{figure*}
\centering
\includegraphics[width=.85\linewidth]{images/attentions/kitti/kitti_2200-class_0-layer_5_compressed.pdf}
\caption{\textbf{Attention maps} for each head at the \textbf{6$^{\rm th}$} (middle) layer of our VaViT-B model trained on \textbf{SemanticKITTI}. The query point, denoted by a red cross, is located on a \textbf{car}. Ground truth in BEV is presented at the top. Subsequent maps have a transparency scaled by the attention weight between the query point and the keys; points with zero attention are fully transparent.}
\label{fig:attention_kitti_car_mid}
\end{figure*}
\begin{figure*}
\centering
\includegraphics[width=.85\linewidth]{images/attentions/kitti/kitti_2200-class_0-layer_11_compressed.pdf}
\caption{\textbf{Attention maps} for each head at the \textbf{12$^{\rm th}$} (last) layer of our VaViT-B model trained on \textbf{SemanticKITTI}. The query point, denoted by a red cross, is located on a \textbf{car}. Ground truth in BEV is presented at the top. Subsequent maps have a transparency scaled by the attention weight between the query point and the keys; points with zero attention are fully transparent.}
\label{fig:attention_kitti_car_last}
\end{figure*}
\begin{figure*}
\centering
\includegraphics[width=.85\linewidth]{images/attentions/waymo/waymo_2200-class_0-layer_0_compressed.pdf}
\caption{\textbf{Attention maps} for each head at the \textbf{1$^{\rm st}$} (first) layer of our VaViT-B model trained on \textbf{WOD}. The query point, denoted by a red cross, is located on a \textbf{car}. Ground truth in BEV is presented at the top. Subsequent maps have a transparency scaled by the attention weight between the query point and the keys; points with zero attention are fully transparent.}
\label{fig:attention_waymo_car_first}
\end{figure*}
\begin{figure*}
\centering
\includegraphics[width=.85\linewidth]{images/attentions/waymo/waymo_2200-class_0-layer_5_compressed.pdf}
\caption{\textbf{Attention maps} for each head at the \textbf{6$^{\rm th}$} (middle) layer of our VaViT-B model trained on \textbf{WOD}. The query point, denoted by a red cross, is located on a \textbf{car}. Ground truth in BEV is presented at the top. Subsequent maps have a transparency scaled by the attention weight between the query point and the keys; points with zero attention are fully transparent.}
\label{fig:attention_waymo_car_mid}
\end{figure*}
\begin{figure*}
\centering
\includegraphics[width=.85\linewidth]{images/attentions/waymo/waymo_2200-class_0-layer_11_compressed.pdf}
\caption{\textbf{Attention maps} for each head at the \textbf{12$^{\rm th}$} (last) layer of our VaViT-B model trained on \textbf{WOD}. The query point, denoted by a red cross, is located on a \textbf{car}. Ground truth in BEV is presented at the top. Subsequent maps have a transparency scaled by the attention weight between the query point and the keys; points with zero attention are fully transparent.}
\label{fig:attention_waymo_car_last}
\end{figure*}
\begin{figure*}
\centering
\includegraphics[width=.85\linewidth]{images/attentions/waymo/waymo_2200-class_6-layer_0_compressed.pdf}
\caption{\textbf{Attention maps} for each head at the \textbf{1$^{\rm st}$} (first) layer of our VaViT-B model trained on \textbf{WOD}. The query point, denoted by a red cross, is located on a \textbf{pedestrian}. Ground truth in BEV is presented at the top. Subsequent maps have a transparency scaled by the attention weight between the query point and the keys; points with zero attention are fully transparent.}
\label{fig:attention_waymo_pedestrian_first}
\end{figure*}
\begin{figure*}
\centering
\includegraphics[width=.85\linewidth]{images/attentions/waymo/waymo_2200-class_6-layer_5_compressed.pdf}
\caption{\textbf{Attention maps} for each head at the \textbf{6$^{\rm th}$} (middle) layer of our VaViT-B model trained on \textbf{WOD}. The query point, denoted by a red cross, is located on a \textbf{pedestrian}. Ground truth in BEV is presented at the top. Subsequent maps have a transparency scaled by the attention weight between the query point and the keys; points with zero attention are fully transparent.}
\label{fig:attention_waymo_pedestrian_mid}
\end{figure*}
\begin{figure*}
\centering
\includegraphics[width=.85\linewidth]{images/attentions/waymo/waymo_2200-class_6-layer_11_compressed.pdf}
\caption{\textbf{Attention maps} for each head at the \textbf{12$^{\rm th}$} (last) layer of our VaViT-B model trained on \textbf{WOD}. The query point, denoted by a red cross, is located on a \textbf{pedestrian}. Ground truth in BEV is presented at the top. Subsequent maps have a transparency scaled by the attention weight between the query point and the keys; points with zero attention are fully transparent.}
\label{fig:attention_waymo_pedestrian_last}
\end{figure*}
% ===============================================
%
% ===============================================
\end{document}
\begin{thebibliography}{63}
\providecommand{\natexlab}[1]{#1}
\providecommand{\url}[1]{\texttt{#1}}
\expandafter\ifx\csname urlstyle\endcsname\relax
\providecommand{\doi}[1]{doi: #1}\else
\providecommand{\doi}{doi: \begingroup \urlstyle{rm}\Url}\fi
\bibitem[Ando et~al.(2023)Ando, Gidaris, Bursuc, Puy, Boulch, and Marlet]{rangevit}
Angelika Ando, Spyros Gidaris, Andrei Bursuc, Gilles Puy, Alexandre Boulch, and Renaud Marlet.
\newblock {RangeViT: Towards Vision Transformers for 3D Semantic Segmentation in Autonomous Driving}.
\newblock In \emph{CVPR}, 2023.
\bibitem[Arnab et~al.(2021)Arnab, Dehghani, Heigold, Sun, Lu{\v{c}}i{\'c}, and Schmid]{arnab2021vivit}
Anurag Arnab, Mostafa Dehghani, Georg Heigold, Chen Sun, Mario Lu{\v{c}}i{\'c}, and Cordelia Schmid.
\newblock Vivit: A video vision transformer.
\newblock In \emph{ICCV}, 2021.
\bibitem[Behley et~al.(2019)Behley, Garbade, Milioto, Quenzel, Behnke, Stachniss, and Gall]{semkitti}
J. Behley, M. Garbade, A. Milioto, J. Quenzel, S. Behnke, C. Stachniss, and J. Gall.
\newblock {SemanticKITTI: A Dataset for Semantic Scene Understanding of LiDAR Sequences}.
\newblock In \emph{ICCV}, 2019.
\bibitem[Boulch et~al.(2020)Boulch, Puy, and Marlet]{fkaconv}
Alexandre Boulch, Gilles Puy, and Renaud Marlet.
\newblock Fkaconv: Feature-kernel alignment for point cloud convolution.
\newblock In \emph{ACCV}, 2020.
\bibitem[Caesar et~al.(2020)Caesar, Bankiti, Lang, Vora, Liong, Xu, Krishnan, Pan, Baldan, and Beijbom]{nuscenes}
Holger Caesar, Varun Bankiti, Alex~H. Lang, Sourabh Vora, Venice~Erin Liong, Qiang Xu, Anush Krishnan, Yu Pan, Giancarlo Baldan, and Oscar Beijbom.
\newblock {nuScenes: A multimodal dataset for autonomous driving}.
\newblock In \emph{CVPR}, 2020.
\bibitem[Caron et~al.(2021)Caron, Touvron, Misra, 'egou, Mairal, Bojanowski, and Joulin1]{dino}
Mathilde Caron, Hugo Touvron, Ishan Misra, Herve~J\ 'egou, Julien Mairal, Piotr Bojanowski, and Armand Joulin1.
\newblock Emerging properties in self-supervised vision transformers.
\newblock In \emph{ICCV}, 2021.
\bibitem[Cheng et~al.(2021)Cheng, Razani, Taghavi, Li, and Liu]{AF2S3Net}
Ran Cheng, Ryan Razani, Ehsan Taghavi, Enxu Li, and Bingbing Liu.
\newblock {(AF)2-S3Net: Attentive Feature Fusion With Adaptive Feature Selection for Sparse Semantic Segmentation Network}.
\newblock In \emph{CVPR}, 2021.
\bibitem[Choe et~al.(2022)Choe, Park, Rameau, Park, and Kweon]{pointmixer}
Jaesung Choe, Chunghyun Park, Francois Rameau, Jaesik Park, and In~So Kweon.
\newblock {PointMixer: MLP-Mixer for Point Cloud Understanding}.
\newblock In \emph{ECCV}, 2022.
\bibitem[Choy et~al.(2019)Choy, Gwak, and Savarese]{minknet}
Christopher Choy, JunYoung Gwak, and Silvio Savarese.
\newblock {4D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networks}.
\newblock In \emph{CVPR}, 2019.
\bibitem[Cortinhal et~al.(2020)Cortinhal, Tzelepis, and Erdal~Aksoy]{salsaNext}
Tiago Cortinhal, George Tzelepis, and Eren Erdal~Aksoy.
\newblock {SalsaNext: Fast, Uncertainty-Aware Semantic Segmentation of LiDAR Point Clouds}.
\newblock In \emph{Advances in Visual Computing}, 2020.
\bibitem[Dehghani et~al.(2023)Dehghani, Djolonga, Mustafa, Padlewski, Heek, Gilmer, Steiner, Caron, Geirhos, Alabdulmohsin, et~al.]{dehghani2023scaling}
Mostafa Dehghani, Josip Djolonga, Basil Mustafa, Piotr Padlewski, Jonathan Heek, Justin Gilmer, Andreas~Peter Steiner, Mathilde Caron, Robert Geirhos, Ibrahim Alabdulmohsin, et~al.
\newblock Scaling vision transformers to 22 billion parameters.
\newblock In \emph{ICML}, 2023.
\bibitem[Dosovitskiy et~al.(2021)Dosovitskiy, Beyer, Kolesnikov, Weissenborn, Zhai, Unterthiner, Dehghani, Minderer, Heigold, Gelly, Uszkoreit, and Houlsby]{dosovitskiy2021image}
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby.
\newblock An image is worth 16x16 words: Transformers for image recognition at scale.
\newblock In \emph{ICLR}, 2021.
\bibitem[Gerdzhev et~al.(2021)Gerdzhev, Razani, Taghavi, and Bingbing]{tornadonet}
Martin Gerdzhev, Ryan Razani, Ehsan Taghavi, and Liu Bingbing.
\newblock {TORNADO-Net: mulTiview tOtal vaRiatioN semAntic segmentation with Diamond inceptiOn module}.
\newblock In \emph{ICRA}, 2021.
\bibitem[Gong et~al.(2021)Gong, Chung, and Glass]{gong2021ast}
Yuan Gong, Yu-An Chung, and James Glass.
\newblock Ast: Audio spectrogram transformer.
\newblock \emph{arXiv preprint arXiv:2104.01778}, 2021.
\bibitem[Heo et~al.(2024)Heo, Park, Han, and Yun]{heo2024ropevit}
Byeongho Heo, Song Park, Dongyoon Han, and Sangdoo Yun.
\newblock Rotary position embedding for vision transformer.
\newblock In \emph{ECCV}, 2024.
\bibitem[Hou et~al.(2022)Hou, Zhu, Ma, Loy, and Li]{PVKD}
Yuenan Hou, Xinge Zhu, Yuexin Ma, Chen~Change Loy, and Yikang Li.
\newblock {Point-to-Voxel Knowledge Distillation for LiDAR Semantic Segmentation}.
\newblock In \emph{CVPR}, 2022.
\bibitem[Huang et~al.(2016)Huang, Sun, Liu, Sedra, and Weinberger]{stocdepth}
Gao Huang, Yu Sun, Zhuang Liu, Daniel Sedra, and Kilian Weinberger.
\newblock {Deep networks with stochastic depth}.
\newblock In \emph{ECCV}, 2016.
\bibitem[Knaebel et~al.(2025)Knaebel, Yilmaz, de~Geus, Hermans, Adrian, Linder, and Leibe]{dinointheroom}
Karim Knaebel, Kadir Yilmaz, Daan de Geus, Alexander Hermans, David Adrian, Timm Linder, and Bastian Leibe.
\newblock Dino in the room: Leveraging 2d foundation models for 3d segmentation.
\newblock \emph{CVPRW}, 2025.
\bibitem[Kochanov et~al.(2020)Kochanov, Nejadasl, and Booij]{KPRNet}
Deyvid Kochanov, Fatemeh~Karimi Nejadasl, and Olaf Booij.
\newblock {KPRNet: Improving projection-based LiDAR semantic segmentation}.
\newblock \emph{arXiv:2007.12668}, 2020.
\bibitem[Kong et~al.(2023{\natexlab{a}})Kong, Liu, Chen, Ma, Zhu, Li, Hou, Qiao, and Liu]{rangeformer}
Lingdong Kong, Youquan Liu, Runnan Chen, Yuexin Ma, Xinge Zhu, Yikang Li, Yuenan Hou, Yu Qiao, and Ziwei Liu.
\newblock Rethinking range view representation for lidar segmentation.
\newblock In \emph{ICCV}, 2023{\natexlab{a}}.
\bibitem[Kong et~al.(2023{\natexlab{b}})Kong, Ren, Pan, and Liu]{lasermix}
Lingdong Kong, Jiawei Ren, Liang Pan, and Ziwei Liu.
\newblock Lasermix for semi-supervised lidar semantic segmentation.
\newblock In \emph{CVPR}, 2023{\natexlab{b}}.
\bibitem[Lai et~al.(2022)Lai, Liu, Jiang, Wang, Zhao, Liu, Qi, and Jia]{strattrans}
Xin Lai, Jianhui Liu, Li Jiang, Liwei Wang, Hengshuang Zhao, Shu Liu, Xiaojuan Qi, and Jiaya Jia.
\newblock {Stratified Transformer for 3D Point Cloud Segmentation}.
\newblock In \emph{CVPR}, 2022.
\bibitem[Lai et~al.(2023)Lai, Chen, Lu, Liu, and Jia]{sphereformer}
Xin Lai, Yukang Chen, Fanbin Lu, Jianhui Liu, and Jiaya Jia.
\newblock {Spherical Transformer for LiDAR-Based 3D Recognition}.
\newblock In \emph{CVPR}, 2023.
\bibitem[Landrieu and Simonovsky(2018)]{SPGraph}
Loic Landrieu and Martin Simonovsky.
\newblock Large-scale point cloud semantic segmentation with superpoint graphs.
\newblock In \emph{CVPR}, 2018.
\bibitem[Li et~al.(2022)Li, Dai, and Ding]{sdseg3d}
Jiale Li, Hang Dai, and Yong Ding.
\newblock Self-distillation for robust lidar semantic segmentation in autonomous driving.
\newblock In \emph{ECCV}, 2022.
\bibitem[{Liong} et~al.(2020){Liong}, {Nguyen}, {Widjaja}, {Sharma}, and {Jie Chong}]{amvnet}
Venice~Erin {Liong}, Thi Ngoc~Tho {Nguyen}, Sergi {Widjaja}, Dhananjai {Sharma}, and Zhuang {Jie Chong}.
\newblock {AMVNet: Assertion-based Multi-View Fusion Network for LiDAR Semantic Segmentation}.
\newblock \emph{arXiv:2012.04934}, 2020.
\bibitem[Liu et~al.(2023)Liu, Yang, Tang, Yang, and Han]{flatformer}
Zhijian Liu, Xinyu Yang, Haotian Tang, Shang Yang, and Song Han.
\newblock Flatformer: Flattened window attention for efficient point cloud transformer.
\newblock In \emph{CVPR}, 2023.
\bibitem[Loshchilov and Hutter(2019)]{adamw}
Ilya Loshchilov and Frank Hutter.
\newblock Decoupled weight decay regularization.
\newblock In \emph{ICLR}, 2019.
\bibitem[Ma et~al.(2022)Ma, Qin, You, Ran, and Fu]{pointmlp}
Xu Ma, Can Qin, Haoxuan You, Haoxi Ran, and Yun Fu.
\newblock {Rethinking Network Design and Local Geometry in Point Cloud: A Simple Residual {MLP} Framework}.
\newblock In \emph{ICLR}, 2022.
\bibitem[Milioto et~al.(2019)Milioto, Vizzo, Behley, and Stachniss]{rangenetpp}
Andres Milioto, Ignacio Vizzo, Jens Behley, and Cyrill Stachniss.
\newblock {RangeNet ++: Fast and Accurate LiDAR Semantic Segmentation}.
\newblock In \emph{IROS}, 2019.
\bibitem[Park et~al.(2022)Park, Jeong, Cho, and Park]{fastpointtrans}
Chunghyun Park, Yoonwoo Jeong, Minsu Cho, and Jaesik Park.
\newblock {Fast Point Transformer}.
\newblock In \emph{CVPR}, 2022.
\bibitem[Park et~al.(2023)Park, Kim, and Soyeong Kim~and]{pcscnet}
Jaehyun Park, Chansoo Kim, and Kichun~Jo Soyeong Kim~and.
\newblock {PCSCNet: Fast 3D semantic segmentation of LiDAR point cloud for autonomous car using point convolution and sparse convolution network}.
\newblock \emph{Expert Systems with Applications}, 2023.
\bibitem[Puy et~al.(2023)Puy, Boulch, and Marlet]{waffleiron}
Gilles Puy, Alexandre Boulch, and Renaud Marlet.
\newblock Using a waffle iron for automotive point cloud semantic segmentation.
\newblock In \emph{ICCV}, 2023.
\bibitem[Puy et~al.(2024)Puy, Gidaris, Boulch, Sim\'eoni, Sautier, P\'erez, Bursuc, and Marlet]{scalr}
Gilles Puy, Spyros Gidaris, Alexandre Boulch, Oriane Sim\'eoni, Corentin Sautier, Patrick P\'erez, Andrei Bursuc, and Renaud Marlet.
\newblock Three pillars improving vision foundation model distillation for lidar.
\newblock In \emph{CVPR}, 2024.
\bibitem[Qi et~al.(2017{\natexlab{a}})Qi, Su, Mo, and Guibas]{pointnet}
Charles~R. Qi, Hao Su, Kaichun Mo, and Leonidas~J. Guibas.
\newblock {PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation}.
\newblock In \emph{CVPR}, 2017{\natexlab{a}}.
\bibitem[Qi et~al.(2017{\natexlab{b}})Qi, Yi, Su, and Guibas]{pointnetpp}
Charles~Ruizhongtai Qi, Li Yi, Hao Su, and Leonidas~J Guibas.
\newblock {PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space}.
\newblock In \emph{NeurIPS}, 2017{\natexlab{b}}.
\bibitem[Qian et~al.(2022)Qian, Li, Peng, Mai, Hammoud, Elhoseiny, and Ghanem]{pointnext}
Guocheng Qian, Yuchen Li, Houwen Peng, Jinjie Mai, Hasan Abed Al~Kader Hammoud, Mohamed Elhoseiny, and Bernard Ghanem.
\newblock {PointNeXt: Revisiting PointNet++ with Improved Training and Scaling Strategies}.
\newblock In \emph{NeurIPS}, 2022.
\bibitem[Qiu et~al.(2022)Qiu, Yu, and Tao]{gfnet}
Haibo Qiu, Baosheng Yu, and Dacheng Tao.
\newblock {GFNet: Geometric Flow Network for 3D Point Cloud Semantic Segmentation}.
\newblock \emph{TMLR}, 2022.
\bibitem[Rist et~al.(2020)Rist, Schmidt, Enzweiler, and Gavrila]{SCSSnet}
Christoph~B. Rist, David Schmidt, Markus Enzweiler, and Dariu~M. Gavrila.
\newblock {SCSSnet: Learning Spatially-Conditioned Scene Segmentation on LiDAR Point Clouds}.
\newblock In \emph{IEEE Intelligent Vehicles Symposium}, 2020.
\bibitem[Robert et~al.(2023)Robert, Raguet, and Landrieu]{superpointtransformer}
Damien Robert, Hugo Raguet, and Loic Landrieu.
\newblock Efficient 3d semantic segmentation with superpoint transformer.
\newblock In \emph{ICCV}, 2023.
\bibitem[Rold{\~a}o et~al.(2020)Rold{\~a}o, de~Charette, and Verroust-Blondet]{lmscnet}
Luis Rold{\~a}o, Raoul de Charette, and Anne Verroust-Blondet.
\newblock {LMSCNet: Lightweight Multiscale 3D Semantic Completion}.
\newblock In \emph{3DV}, 2020.
\bibitem[Su et~al.(2023)Su, Ahmed, Lu, Pan, Bo, and Liu]{rope}
Jianlin Su, Murtadha Ahmed, Yu Lu, Shengfeng Pan, Wen Bo, and Yunfeng Liu.
\newblock Roformer: Enhanced transformer with rotary position embedding.
\newblock \emph{Neurocomputing}, 2023.
\bibitem[Sun et~al.(2020)Sun, , Kretzschmar, Dotiwalla, Chouard, Patnaik, Tsui, Guo, Zhou, Chai, Caine, Vasudevan, Han, Ngiam, Zhao, Timofeev, Ettinger, Krivokon, Gao, Joshi, Zhao, Cheng, Zhang, Shlens, Chen, and Anguelov]{waymo}
Pei Sun, , Henrik Kretzschmar, Xerxes Dotiwalla, Aur\'elien Chouard, Vijaysai Patnaik, Paul Tsui, James Guo, Yin Zhou, Yuning Chai, Benjamin Caine, Vijay Vasudevan, Wei Han, Jiquan Ngiam, Hang Zhao, Aleksei Timofeev, Scott Ettinger, Maxim Krivokon, Amy Gao, Aditya Joshi, Sheng Zhao, Shuyang Cheng, Yu Zhang, Jonathon Shlens, Zhifeng Chen, and Dragomir Anguelov.
\newblock Scalability in perception for autonomous driving: Waymo open dataset.
\newblock In \emph{CVPR}, 2020.
\bibitem[Tang et~al.(2020)Tang, Liu, Zhao, Lin, Lin, Wang, and Han]{spvnas}
Haotian Tang, Zhijian Liu, Shengyu Zhao, Yujun Lin, Ji Lin, Hanrui Wang, and Song Han.
\newblock Searching efficient 3d architectures with sparse point-voxel convolution.
\newblock In \emph{ECCV}, 2020.
\bibitem[Thomas et~al.(2019)Thomas, Qi, Deschaud, Marcotegui, Goulette, and Guibas]{kpconv}
Hugues Thomas, Charles~R. Qi, Jean-Emmanuel Deschaud, Beatriz Marcotegui, Francois Goulette, and Leonidas~J. Guibas.
\newblock {KPConv: Flexible and Deformable Convolution for Point Clouds}.
\newblock In \emph{ICCV}, 2019.
\bibitem[Tolstikhin et~al.(2021)Tolstikhin, Houlsby, Kolesnikov, Beyer, Zhai, Unterthiner, Yung, Steiner, Keysers, Uszkoreit, Lucic, and Dosovitskiy]{mlpmixer}
Ilya~O Tolstikhin, Neil Houlsby, Alexander Kolesnikov, Lucas Beyer, Xiaohua Zhai, Thomas Unterthiner, Jessica Yung, Andreas Steiner, Daniel Keysers, Jakob Uszkoreit, Mario Lucic, and Alexey Dosovitskiy.
\newblock {MLP-Mixer: An all-MLP Architecture for Vision}.
\newblock In \emph{NeurIPS}, 2021.
\bibitem[Vaswani et~al.(2017)Vaswani, Shazeer, Parmar, Uszkoreit, Jones, Gomez, Kaiser, and Polosukhin]{vaswani2017attention}
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan~N Gomez, {\L}ukasz Kaiser, and Illia Polosukhin.
\newblock Attention is all you need.
\newblock \emph{NeurIPS}, 2017.
\bibitem[Wang et~al.(2019)Wang, Sun, Liu, Sarma, Bronstein, and Solomon]{dgcnn}
Yue Wang, Yongbin Sun, Ziwei Liu, Sanjay~E Sarma, Michael~M Bronstein, and Justin~M Solomon.
\newblock Dynamic graph cnn for learning on point clouds.
\newblock \emph{ACM Transactions On Graphics}, 2019.
\bibitem[Wu et~al.(2024{\natexlab{a}})Wu, Jiang, Wang, Liu, Liu, Qiao, Ouyang, He, and Zhao]{ptv3}
Xiaoyang Wu, Li Jiang, Peng-Shuai Wang, Zhijian Liu, Xihui Liu, Yu Qiao, Wanli Ouyang, Tong He, and Hengshuang Zhao.
\newblock Point transformer v3: Simpler faster stronger.
\newblock In \emph{CVPR}, 2024{\natexlab{a}}.
\bibitem[Wu et~al.(2024{\natexlab{b}})Wu, Peng, Xie, Hou, Lin, Huang, Liu, Cai, and Ouyang]{pillarmix}
Xiaopei Wu, Liang Peng, Liang Xie, Yuenan Hou, Binbin Lin, Xiaoshui Huang, Haifeng Liu, Deng Cai, and Wanli Ouyang.
\newblock Semi-supervised 3d object detection with patchteacher and pillarmix.
\newblock In \emph{AAAI}, 2024{\natexlab{b}}.
\bibitem[Xiao et~al.(2022)Xiao, Huang, Guan, Cui, Lu, and Shao]{polarmix}
Aoran Xiao, Jiaxing Huang, Dayan Guan, Kaiwen Cui, Shijian Lu, and Ling Shao.
\newblock {PolarMix: A General Data Augmentation Technique for Li{DAR} Point Clouds}.
\newblock In \emph{NeurIPS}, 2022.
\bibitem[Xu et~al.(2020)Xu, Wu, Wang, Zhan, Vajda, Keutzer, and Tomizuka]{SqueezeSegV3}
Chenfeng Xu, Bichen Wu, Zining Wang, Wei Zhan, Peter Vajda, Kurt Keutzer, and Masayoshi Tomizuka.
\newblock {SqueezeSegV3: Spatially-Adaptive Convolution for Efficient Point-Cloud Segmentation}.
\newblock In \emph{ECCV}, 2020.
\bibitem[Xu et~al.(2021)Xu, Zhang, Dou, Zhu, Sun, and Pu]{RPVNet}
Jianyun Xu, Ruixiang Zhang, Jian Dou, Yushi Zhu, Jie Sun, and Shiliang Pu.
\newblock {RPVNet: A Deep and Efficient Range-Point-Voxel Fusion Network for LiDAR Point Cloud Segmentation}.
\newblock In \emph{ICCV}, 2021.
\bibitem[Xu et~al.(2023)Xu, Zhu, and Clifton]{xu2023multimodal}
Peng Xu, Xiatian Zhu, and David~A Clifton.
\newblock Multimodal learning with transformers: A survey.
\newblock \emph{IEEE Transactions on Pattern Analysis and Machine Intelligence}, 2023.
\bibitem[Yan et~al.(2022)Yan, Gao, Zheng, Zheng, Zhang, Cui, and Li]{2dpass}
Xu Yan, Jiantao Gao, Chaoda Zheng, Chao Zheng, Ruimao Zhang, Shuguang Cui, and Zhen Li.
\newblock {2DPASS: 2D Priors Assisted Semantic Segmentation on LiDAR Point Clouds}.
\newblock In \emph{ECCV}, 2022.
\bibitem[Ye et~al.(2022)Ye, Wan, Shuangjie~Xu, and Chen]{gasn}
Maosheng Ye, Rui Wan, Tongyi~Cao Shuangjie~Xu, and Qifeng Chen.
\newblock {Efficient Point Cloud Segmentation with Geometry-Aware Sparse Networks}.
\newblock In \emph{ECCV}, 2022.
\bibitem[Yue et~al.(2026)Yue, Robert, Wang, Hong, Wegner, Rupprecht, and Schindler]{litept}
Yuanwen Yue, Damien Robert, Jianyuan Wang, Sunghwan Hong, Jan~Dirk Wegner, Christian Rupprecht, and Konrad Schindler.
\newblock Litept: Lighter yet stronger point transformer.
\newblock \emph{CVPR}, 2026.
\bibitem[Zhai et~al.(2022)Zhai, Kolesnikov, Houlsby, and Beyer]{zhai2022scaling}
Xiaohua Zhai, Alexander Kolesnikov, Neil Houlsby, and Lucas Beyer.
\newblock Scaling vision transformers.
\newblock In \emph{CVPR}, 2022.
\bibitem[Zhang et~al.(2020{\natexlab{a}})Zhang, Fang, Wah, and Torr]{fusionnet}
Feihu Zhang, Jin Fang, Benjamin Wah, and Philip Torr.
\newblock {Deep FusionNet for Point Cloud Semantic Segmentation}.
\newblock In \emph{ECCV}, 2020{\natexlab{a}}.
\bibitem[Zhang et~al.(2020{\natexlab{b}})Zhang, Zhou, David, Yue, Xi, Gong, and Foroosh]{Polarnet}
Yang Zhang, Zixiang Zhou, Philip David, Xiangyu Yue, Zerong Xi, Boqing Gong, and Hassan Foroosh.
\newblock {PolarNet: An Improved Grid Representation for Online LiDAR Point Clouds Semantic Segmentation}.
\newblock In \emph{CVPR}, 2020{\natexlab{b}}.
\bibitem[Zhao et~al.(2021)Zhao, Jiang, Jia, Torr, and Koltun]{pointtransformer}
Hengshuang Zhao, Li Jiang, Jiaya Jia, Philip~H.S. Torr, and Vladlen Koltun.
\newblock {Point Transformer}.
\newblock In \emph{ICCV}, 2021.
\bibitem[Zhao et~al.(2022)Zhao, Xu, Liu, Ming, and Tao]{svaseg}
Lin Zhao, Siyuan Xu, Liman Liu, Delie Ming, and Wenbing Tao.
\newblock {SVASeg: Sparse Voxel-Based Attention for 3D LiDAR Point Cloud Semantic Segmentation}.
\newblock \emph{Remote Sens.}, 2022.
\bibitem[Zhu et~al.(2021)Zhu, Zhou, Wang, Hong, Ma, Li, Li, and Lin]{Cylinder3d}
Xinge Zhu, Hui Zhou, Tai Wang, Fangzhou Hong, Yuexin Ma, Wei Li, Hongsheng Li, and Dahua Lin.
\newblock {Cylindrical and Asymmetrical 3D Convolution Networks for LiDAR Segmentation}.
\newblock In \emph{CVPR}, 2021.
\end{thebibliography}