Johnson-Lindenstrauss Lemma Guided Network for Efficient 3D Medical Segmentation
Extracted source text · arXiv 2509.22307
\documentclass{article} % For LaTeX2e
\usepackage{iclr2026_conference,times}
% Optional math commands from https://github.com/goodfeli/dlbook_notation.
\input{math_commands.tex}
\usepackage{hyperref}
\usepackage{url}
\usepackage{amsmath,amssymb,amsfonts}
\usepackage{graphicx}
\usepackage{textcomp}
\usepackage{array}
\usepackage[caption=false,font=normalsize,labelfont=sf,textfont=sf]{subfig}
\usepackage{stfloats}
\usepackage{url}
\usepackage{verbatim}
\usepackage{multirow}
\usepackage{marvosym}
\let\proof\relax
\let\endproof\relax
\usepackage{amsthm}
\usepackage{enumerate}
\usepackage{threeparttable}
\usepackage{adjustbox}
\usepackage{booktabs}
\usepackage{wrapfig}
\usepackage{caption}
\usepackage{marvosym}
% \newtheorem{assumption}{Assumption}
% \newtheorem{lemma}{Lemma}
% \newtheorem{theorem}{Theorem}
\newtheorem{assumption}{Assumption}
\newtheorem{lemma}{Lemma}
\newtheorem{theorem}{Theorem}
\newtheorem{definition}{Definition}
\newtheorem{remark}{Remark}
\newtheorem{corollary}[theorem]{Corollary}
\usepackage[table,dvipsnames]{xcolor}
\definecolor{dr}{HTML}{b71a3b}
\definecolor{dg}{HTML}{0aa344}
\definecolor{db}{rgb}{0.0, 0.3, .9}
\definecolor{line_blue}{HTML}{1f77b4}
\definecolor{line_orange}{HTML}{ff7f0e}
\definecolor{line_green}{HTML}{2ca02c}
\definecolor{line_red}{HTML}{d62728}
\definecolor{method_brown}{HTML}{843F0B}
\definecolor{method_blue}{HTML}{1E386B}
\definecolor{table_green}{rgb}{0.89, 0.93, 0.85}
\definecolor{caption_green}{HTML}{CBDEB8}
\newcommand{\gr}{\cellcolor{table_green}}
\usepackage[ruled,vlined]{algorithm2e}
% --- Custom Colors ---
% A distinct green for comments
\definecolor{codegreen}{rgb}{0,0.5,0.1}
% A clear blue for function names
\definecolor{funcblue}{rgb}{0.1,0.2,0.7}
% --- Custom Commands & Styles ---
% Set the comment style to use the new color
\newcommand\mycommfont[1]{\footnotesize\ttfamily\textcolor{codegreen}{#1}}
\SetCommentSty{mycommfont}
% Command for styling function names with color and sans-serif font
\newcommand{\func}[1]{\textcolor{funcblue}{\textsf{#1}}}
\usepackage[accsupp]{axessibility}
\title{Johnson-Lindenstrauss Lemma Guided Network for Efficient 3D Medical Segmentation}
% Authors must not appear in the submitted version. They should be hidden
% as long as the \iclrfinalcopy macro remains commented out below.
% Non-anonymous submissions will be rejected without review.
% \author{Anonymous submission to ICLR 2026}
\author{Jinpeng Lu\text{$^{1}$\thanks{~Equal contribution. Correspondence to Zhiwei Xiong}},\quad Linghan Cai\text{$^{2*}$},\quad Yinda Chen\text{$^{1*}$},\quad Guo Tang\text{$^{3}$},\quad Songhan Jiang\text{$^{3}$},\\
\quad \textbf{Haoyuan Shi\text{$^{1}$}},\quad \textbf{Zhiwei Xiong\text{$^{1}$}\textsuperscript{\Letter}}
\vspace{0.3em} \\
\text{$^{1}$} University of Science and Technology of China\\
\text{$^{2}$} Dresden University of Technology\\
\text{$^{3}$} Harbin Institute of Technology, Shenzhen\\
\texttt{jinplu@mail.ustc.edu.cn, zwxiong@ustc.edu.cn} \\
}
% The \author macro works with any number of authors. There are two commands
% used to separate the names and addresses of multiple authors: \And and \AND.
%
% Using \And between authors leaves it to \LaTeX{} to determine where to break
% the lines. Using \AND forces a linebreak at that point. So, if \LaTeX{}
% puts 3 of 4 authors names on the first line, and the last on the second
% line, try using \AND instead of \And before the third author name.
\newcommand{\fix}{\marginpar{FIX}}
\newcommand{\new}{\marginpar{NEW}}
\iclrfinalcopy
%\iclrfinalcopy % Uncomment for camera-ready version, but NOT for submission.
\begin{document}
\maketitle
\begin{abstract}
Lightweight 3D medical image segmentation remains constrained by a fundamental \textit{``efficiency / robustness conflict''}, particularly when processing complex anatomical structures and heterogeneous modalities. In this paper, we study how to redesign the framework based on the characteristics of high-dimensional 3D images, and explore data synergy to overcome the fragile representation of lightweight methods. Our approach, VeloxSeg, begins with a deployable and extensible dual-stream CNN-Transformer architecture composed of Paired Window Attention (PWA) and Johnson-Lindenstrauss lemma-guided convolution (JLC). For each 3D image, we invoke a ``glance-and-focus'' principle, where PWA rapidly retrieves multi-scale information, and JLC ensures robust local feature extraction with minimal parameters, significantly enhancing the model's ability to operate with low computational budget. Followed by an extension of the dual-stream architecture that incorporates modal interaction into the multi-scale image-retrieval process, VeloxSeg efficiently models heterogeneous modalities. Finally, Spatially Decoupled Knowledge Transfer (SDKT) via Gram matrices injects the texture prior extracted by a self-supervised network into the segmentation network, yielding stronger representations than baselines at no extra inference cost. Experimental results on multimodal benchmarks show that VeloxSeg achieves a 26\% Dice improvement, alongside increasing GPU throughput by 11$\times$ and CPU by 48$\times$. Codes are available at \url{https://github.com/JinPLu/VeloxSeg}.
\end{abstract}
\section{Introduction}
3D medical image segmentation serves as a cornerstone of contemporary clinical workflows~\citep{wu2025towards, peiris2023uncertainty}, driving rapid advances in semantic segmentation models~\citep{vsmtrans, unetr++, he2025vista3d, yu20253d, wald2025revisiting}. However, translating these advances into clinical practice faces significant obstacles, including limited hardware resources, stringent latency requirements, and the need to achieve multi-organ generalization while handling heterogeneous multimodal data in deployment environments. These challenges have spurred the development of lightweight 3D medical segmentation methods, leading to lightweight approaches with fewer than 5 million parameters~\citep{segformer3d, slimunetr, superlightnet, hcma_unet, u_rwkv}. Yet, the pursuit of smaller parameter counts and lower computational costs has revealed a fundamental and increasingly prominent trade-off: these lightweight models struggle to maintain both efficiency and robust performance when handling heterogeneous data and complex lesions, which we term \textit{``efficiency / robustness conflict''}. We address this problem from two key perspectives:
\begin{figure}[t]
\centering
\includegraphics[width=\textwidth]{figure/dice_throughput.pdf}
\caption{Comparison of our proposed VeloxSeg with recent methods on the AutoPET-II test set. Basic models, multimodal models, lightweight models, and our model are marked with \textcolor{line_blue}{circles}, \textcolor{line_orange}{squares}, \textcolor{line_green}{triangles}, and \textcolor{line_red}{stars}, respectively. GPU and CPU Throughput are measured on an NVIDIA RTX3090 GPU and a single-core Intel(R) Xeon(R) Gold 5320 CPU, respectively.}
\label{fig:dice_thr}
\vspace{-0.5cm}
\end{figure}
\textbf{Insufficient consideration of the high-dimensional complexity of 3D data.} Recent sequence models, such as Mamba~\citep{mamba, xing2025segmamba} and RWKV~\citep{rwkv, u_rwkv}, have achieved remarkable progress in segmentation, owing to their linear complexity and long-range modeling capabilities. However, due to the lack of more efficient scanning strategies suitable for 3D data, these methods have not yet supplanted CNN-Transformer architectures in the domain of efficient medical segmentation. Our model is built on a dual-stream CNN-Transformer architecture, synergizing the complementary strengths of both components: the inductive bias~\citep{lecun_cnn, inductive_bias} and training stability~\citep{resnet, batchnorm, instancenorm} of convolutions with the global modeling power~\citep{attention_is_all_you_need} and extensibility~\citep{attn_text_image1, attn_text_image2, attn_text_image3} of Transformers. Pruning is the most common approach to model lightweighting~\citep{taylor_importance, torch_pruning}, but its final configuration relies on dataset-specific importance metrics and hand-tuned sparsity schedules, which limit generalizability and lead to expensive retraining. Therefore, developing lightweight and efficient components is essential. Despite this mature design, existing 3D attention mechanisms face a fundamental trade-off between capturing local-global context and managing computational costs. To draw an analogy, their computational costs are like the total time a team of detectives spends searching for clues. Window-based attention~\citep{swinunetr, du2025tdformer} is the meticulous detective, poring over every detail but proceeding slowly. Axial-based~\citep{vsmtrans} and downsampling-based~\citep{slimunetr, segformer3d, kuang2025lw} attention are the hasty profilers, quickly forming a broad picture but often missing pivotal evidence. Our Paired Window Attention (PWA) overcomes their limitations by assembling a team of diverse detectives. By deploying parallel feature streams to search key information across multiple scales at once, it's as if the team works in concert and then shares their unique perspectives, ``glancing'' the key clues in a shorter time. Furthermore, the detailed analysis of each clue depends on the local modeling capabilities of lightweight convolutions. The dominant approach, depth-wise separable design~\citep{xception, shufflenetv2, mednext, muhammad2025towards}, suffers from a critical flaw: its channel decoupling fragments features, shattering the geometric adjacency of data in the feature space. To continue the analogy, this prevents detectives from connecting a new clue to crucial memories of the case. This breakdown is especially severe when confronted with complex anatomical structures and heterogeneous modalities. Our Johnson–Lindenstrauss (JL) lemma-guided lightweight convolution~\citep{JL_Lemma} tackles this directly. By providing a lower bound on the group size that preserves spatial adjacency for convolutions at each stage of the network, we ensure that every detail can be robustly extracted while remaining lightweight.
\textbf{Insufficient exploration of data synergy, including multimodal cooperation and data priors.} Exploiting multimodal complementary information is crucial for robust model representation~\citep{pet/ct1, zheng2025asymmetric, zou2025mmr}. However, it is often ignored by lightweight methods due to the potential increase in computational cost, even when training on multimodal datasets. As discussed in Appendix~\ref{appendix:characteristics_medical_multi_modality}, bridging multimodal information across multiple scales is vital for extracting complementary information from heterogeneous modalities. Therefore, we extend our dual-stream architecture, using PWA to facilitate efficient modal interaction at the additional cost of only 0.27 MParams and 0.09 GFLOPs. Besides, exploring prior knowledge from existing data to enhance a model's detailed representation holds practical significance for efficient segmentation methods. These methods often achieve higher efficiency by performing segmentation in a compressed space, which comes at the cost of exploring small lesions and complex boundaries~\citep{segformer3d, h2former, emcad, slimunetr}. Although establishing cross-task knowledge transfer from reconstruction to segmentation appears to be a solution~\citep{multi_task_theory, rui2025multi, auxiliary, masktwins}, the significant differences in their regions of interest (ROIs) often lead to negative knowledge transfer~\citep{ds2f}. To this end, our proposed Spatially Decoupled Knowledge Transfer (SDKT) is a simple yet effective solution, motivated by the observation that a common upsampling operation in reconstruction and super-resolution tasks, ``Conv+PixelShuffle~\citep{review_superresolution}'', essentially unfolds the channel relationships at each voxel position into the spatial details of the surrounding image patch. This suggests that the guidance provided by a texture teacher to a segmentation task should be based on the channel relationships within its features. The Gram matrix, commonly used to represent style in the field of image style transfer, characterizes feature channel relationships in a spatially-invariant manner~\citep{gram1, gram2}. Based on it, we establish a positive knowledge transfer path from a self-supervised texture teacher to the segmentation network with no inference overhead~\citep{texture, ssl_texture}.
Inspired by the above insights, we propose VeloxSeg that systematically alleviates the ``\textit{efficiency / robustness conflict}'' during model lightweighting. Extensive experiments thoroughly explored the rationale for the design choices and demonstrated the model's excellent clinical applicability and generalization capabilities. Figure~\ref{fig:dice_thr} shows a comparison of VeloxSeg's performance with other methods on the AutoPET-II~\citep{AutoPET-II} dataset, demonstrating strong competitiveness. In summary, we develop:
\begin{itemize}
\item A Paired Window Attention to ensemble multi-scale attention groups, capturing local-global information simultaneously, improving localization capabilities with less cost, and achieving low-cost but effective modal interaction at multiple scales.
\item A Johnson-Lindenstrauss lemma-guided convolution that theoretically determines a minimum group size to preserve spatial adjacency, ensuring robust local feature extraction without costly and data-specific pruning.
\item A Spatially Decoupled Knowledge Transfer strategy that uses Gram matrices to distill rich textural details from a self-supervised teacher during training, enhancing model fidelity with zero inference overhead.
\end{itemize}
\begin{figure}[t]
\centering
\includegraphics[width=\textwidth]{figure/Overview.pdf}
\caption{Overview of VeloxSeg. VeloxSeg employs an encoder-decoder architecture with Paired Window Attention (PWA) and Johnson-Lindenstrauss lemma-guided convolution (JLC) on the left, using 1$\times$1 convolution as modal mixer. GC: group convolution; GA: multimodal grouped attention.}
\label{fig:overview}
\vspace{-0.4cm}
\end{figure}
\section{Methodology}
\subsection{Overview of VeloxSeg}
\label{method: Overview}
\begin{figure}[t]
\centering
\includegraphics[width=\textwidth]{figure/Method.pdf}
\caption{\textcolor{method_brown}{\textbf{(a)}} Overview of Paired Window Attention (PWA). \textcolor{method_blue}{\textbf{(b)}} Intuitive difference between depth-wise (DW) convolution and Johnson-Lindenstrauss (JL) guided Convolution in the feature space.}
\label{fig:method}
\end{figure}
As shown in Figure~\ref{fig:overview}, VeloxSeg employs two 4-stage encoders, a modal-fusion convolution encoder and a modal-cooperative Transformer encoder, along with a segmentation decoder and a detail texture teacher. The Paired Window Attention (PWA), a key component of the transformer encoder, is designed to capture multi-scale and cross-modal context with low enough cost. The Johnson-Lindenstrauss lemma-guided Convolution (JLC), a key component of the convolution encoder, consists of 3 parallel JLCs at different scales to fuse modal information and model local features. Separating these two avoids a parameter explosion as the number of modalities increases, while maximizing the advantages and parallelism of both. In training, the Spatially Decoupled Knowledge Transfer (SDTK) strategy is used to enhance texture representation, which is also of great significance for super-resolution and segmentation tasks.
\subsection{Paired Window Attention}
To achieve sufficiently strong clue-capturing capabilities with minimal computational cost, PWA ensembles parallel feature streams to capture key multimodal information at multiple scales. Notably, our approach differs significantly from conventional parallel multi-attention approaches~\citep{vsmtrans, unetr++}, aiming to create a faster, lower-cost, more effective, and more elegant feature stream. Given $M$ modal features from $k$-th stage $\mathbf{E}_m^k, m=1,\cdots,M$, they are first projected into $\mathbf{Q}_m^k,\mathbf{K}_m^k,\mathbf{V}_m^k$. As shown in Figure~\ref{fig:method} (a), we (i) partition the features into a set of big windows, collecting a salient token for each small window; (ii) synchronously expand window pairs to obtain multimodal sequences $\mathbf{X}_{m,i}^k,\mathbf{X}\in\left\{\mathbf{Q},\mathbf{K},\mathbf{V}\right\}$ of different scales but equal length, where $i$ is the number of the paired window; (iii) gather all sequences and compute attention across all scales and modalities at once; and (iv) use a lightweight mixer to simply and efficiently blend features from all scales. The attention $\mathbf{A}_{m}^{k}$ is obtained by the following formula:
\begin{equation}
\mathbf{A}_{m}^{k} = \text{PWA}\left(\mathbf{E}_{m}^k\mid\mathbf{E}_{1}^k,\cdots,\mathbf{E}_{M}^k\right).
\end{equation}
For more information about PWA, please see the Appendix~\ref{appendix:details_PWA}, including PyTorch code, detailed formula flow, and complexity analysis. We also provide a detailed analysis of the necessity of multi-scale modeling of medical modalities. Notably, PWA requires only $\log(\textit{size}), \text{\textit{size}} \in \{H, W, D\}$ paired windows to capture global context, while the minimum window ensures the preservation of local details. PWA achieves near-linear complexity, with a linear coefficient of approximately 7.87\% of Swin Transformer~\citep{swint}.
\subsection{Johnson-Lindenstrauss Lemma-guided Convolution}
\label{method: JLC}
\begin{lemma}[Johnson-Lindenstrauss]
\label{lemma: JL}
For any finite set $\mathcal{X}\subset\mathbb{R}^d$ with $|\mathcal{X}|=N$ and $\varepsilon\in(0,1)$, there exists a linear map $f:\mathbb{R}^d\to\mathbb{R}^{d^\prime}$ with $d^\prime\ge c_{\text{JL}}\,\varepsilon^{-2}\log N$, all $x,y\in\mathcal{X}$ satisfy
$(1-\varepsilon)\|x-y\|_2 \le \|f(x)-f(y)\|_2 \le (1+\varepsilon)\|x-y\|_2$.
\end{lemma}
As shown in Figure~\ref{fig:method} (b), depth-wise convolution destroys the adjacency relationship between data in the feature space, making it difficult to connect the current clues with the key information of the case. Inspired by~\citet{jl_attention} derivation of the minimum attention head size via the Johnson-Lindenstrauss (JL) lemma, we build our lightest but robust convolution upon the above theory framework. In particular, we extend it to the 3D segmentation, exploring the lower bound on group size while preserving spatial adjacency. The JL lemma states that for $N$ points in high-dimensional space, we need at least $\mathcal{O}(\log N)$ embedding dimensions to preserve pairwise distances. The volume ratio of the input image of $M$ modalities to the intermediate feature is $v$, and each voxel of the feature must retain information from at least $v$ input voxels. Due to anatomical constraints and the boundedness of the normalized input values, the manifold $\mathcal{M}$ of segmentation-related information of the input image patch can be covered by a finite number of samples, with a coverage count of $N(M, v)$. Substituting $N=N(M, v)$ into the lemma yields the size of the convolution group:
\begin{equation}
C_{\mathrm{group}} = d^\prime \geq c_{\text{JL}} \varepsilon^{-2} \log N(M, v),
\label{eq:jl_bound}
\end{equation}
where $C_{\mathrm{group}}$ is the number of channels per group.
Due to the lack of $N$ in the vision domain, we empirically approximate $N(M,v)$ using $\hat{N}(M,v)=(M\cdot v)^{\alpha}$, where $\alpha$ is related to the difficulty of the segmentation task at hand. We conduct ablation studies on datasets with the richest modality heterogeneity and data distribution to identify the most generalizable scaling factor, which we use to obtain a lower bound on the group size of the convolution layers in each network stage. As analyzed in the Appendix~\ref{appendix:detials_jl_group_size}, we will use $\left\{C_{\mathrm{group}}^k\right\}_{k=1}^4 = \left\{n,2n,2n,4n\right\}$ as the group size for each stage of our network, where $n\in \mathbb{N}$ is determined from the most challenging AutoPET-II~\citep{AutoPET-II} dataset to ensure multi-organ generalization capability.
\subsection{Spatially Decoupled Knowledge Transfer}
To strengthen the representation of the lightweight model, we transfer the rich texture details extracted by the self-supervised texture teacher to the segmentation network via the Gram matrix. Specifically, we start with learning $M$ self-supervised detail texture teachers $T_m, m=1,\cdots, M$, who are optimized by $M$ reconstruction tasks. The Gram matrix is commonly used to represent image style and can capture feature channel relationships in a spatially invariant manner. For feature maps $\mathbf{X} \in \mathbb{R}^{C \times \left(HWD\right)}$ with $C$ channels, the Gram matrix is:
\begin{equation}
\mathrm{GM}(\mathbf{X}) =\frac{1}{CHWD}\left(\mathbf{X}\mathbf{X}^T\right) \in \mathbb{R}^{C \times C}.
\end{equation}
SDKT is implemented by matching Gram matrices, which is mathematically equivalent to minimizing the maximum mean difference (MMD) using a second-order polynomial kernel~\citep{gram_math1, gram_math2}. This naturally avoids a series of issues caused by excessive ROI discrepancies between the reconstructed/super-resolution features and the segmentation features, while also being more interpretable. Specifically, a Gram-based consistency constraint serves as a positive knowledge transfer path between the segmentation features and the $M$ teacher features:
\begin{equation}
\mathcal{L}_{sdkt}=\sum_{m=1}^M{w_{T}^m\left\| \mathrm{GM}\left( \mathbf{D}_{T}^{m} \right) -\mathrm{GM}\left( \mathbf{D}_{seg} \right) \right\| ^2},
\end{equation}
where $\mathbf{D}_{T}^m$ and $\mathbf{D}_{seg}$ denote the features from the $T_m$ and segmentation decoder, $w_{T}^m$ is loss weight.
\section{Experiments}
\subsection{Datasets \& Metrics}
We validate the effectiveness of VeloxSeg on four public datasets: AutoPET-II~\citep{AutoPET-II}, Hecktor2022~\citep{Hecktor2022}, BraTS2021~\citep{BraTS2021}, and MSD2019 Task-01~\citep{MSD2019} (details in Appendix~\ref{appendix:dataset_details}). Unlike typical medical segmentation datasets, the modality heterogeneity of PET/CT and the complex anatomical structures of multiple organs, and even the whole body, pose unique challenges to all models. We adopt comprehensive evaluation metrics suitable for clinical settings: Model Size (MParams), Computational Complexity (GFLOPs), Efficiency (GPU/CPU Throughput), and Segmentation Performance measured by Dice similarity coefficient (Dice) as the primary indicator, alongside 95\% Hausdorff distance (HD95), Precision, and Recall.
\subsection{Implementation Details \& Baselines}
\label{sec:implementation_details_baseline}
Our implementation is based on PyTorch 2.4.1. Training is performed on an NVIDIA GeForce RTX 3090 GPU, while inference is run on an Intel(R) Xeon(R) Gold 5320 CPU. All datasets are standardized and partitioned into training, validation, and testing subsets in a 6:2:2 ratio. For training, we use a batch size of 4 with a 1:1 positive-to-negative sample ratio. Data augmentation involves random z-axis flipping with a 0.5 probability. We train the model for 300 epochs using the AdamW optimizer~\citep{adamw} with an initial learning rate of 2.5e-4 and a weight decay of 0.01. The learning rate is managed by a linear warmup and cosine annealing scheduler~\citep{warmup_cosine}.
To ensure a convincing evaluation, we benchmark our method against a diverse set of models, including 7 basic models, 3 multimodal models, and 5 lightweight models, which are categorized accordingly in Tables~\ref{tab:autopetii_hecktor2022_computation_performance}. Furthermore, our analysis covers five distinct architectural paradigms: CNN-based models (UNet~\citep{unet3d}, VNet~\citep{vnet}, A2FSeg~\citep{a2fseg}); CNN-Transformer hybrids (UNETR~\citep{unetr}, Nestedformer~\citep{nestedformer}, SuperLightNet~\citep{superlightnet}); CNN-KAN hybrids (U-KAN~\citep{kan}); CNN-Mamba hybrids (HCMA-UNet~\citep{hcma_unet}); and CNN-RWKV hybrids (U-RWKV~\citep{u_rwkv}). The comparison is extended to include 2 advanced vision foundation models: SAM-Med3D~\citep{sam_med3d}, which is evaluated in a zero-shot setting, and DINOv3~\citep{dinov3}, for which the linear head is fine-tuned~\citep{liu2025does}.
\subsection{Clinical Capabilities Evaluation}
\begin{table}[t]
\centering
\small
\setlength{\tabcolsep}{3pt}
\renewcommand{\arraystretch}{1.2}
\begin{threeparttable}
\begin{tabular}{l l c c c c c c c c}
\toprule
\multirow{2}{*}{\textbf{Method}} & \multirow{2}{*}{\textbf{Venue}} & \multicolumn{4}{c}{\textbf{AutoPET-II}} & \multicolumn{4}{c}{\textbf{Hecktor2022}} \\
\cmidrule(lr){3-6} \cmidrule(lr){7-10}
& & \textbf{Dice} $\uparrow$ & \textbf{HD95} $\downarrow$ & \textbf{Prec.} $\uparrow$ & \textbf{Rec.} $\uparrow$ & \textbf{Dice} $\uparrow$ & \textbf{HD95} $\downarrow$ & \textbf{Prec.} $\uparrow$ & \textbf{Rec.} $\uparrow$ \\
\midrule
UNet & MICCAI'16 & $59.41$ & $241.31$ & $62.32$ & $70.74$ & $50.25$ & $65.03$ & $72.13$ & $41.50$ \\
VNet & 3DV'15 & $53.21$ & $242.78$ & $53.21$ & $60.85$ & $55.61$ & $41.46$ & $\textcolor{db}{\mathbf{78.21}}$ & $46.01$ \\
UNETR & WACV'22 & $51.49$ & $257.30$ & $51.49$ & $61.03$ & $48.10$ & $73.27$ & $70.71$ & $39.11$ \\
Swin UNETR & MICCAI'21 & $62.24$ & $242.07$ & $62.91$ & $73.30$ & $44.56$ & $103.02$ & $62.43$ & $37.55$ \\
VSmTrans & MIA'24 & $\textcolor{db}{\mathbf{62.46}}$ & $223.88$ & $65.19$ & $70.92$ & $52.91$ & $78.03$ & $61.91$ & $\textcolor{dr}{\mathbf{50.97}}$ \\
UNETR++ & TMI'24 & $36.50$ & $178.57$ & $36.50$ & $60.16$ & $29.95$ & $27.74$ & $61.84$ & $21.75$ \\
U-KAN & AAAI'25 & $60.67$ & $70.91$ & $62.03$ & $72.94$ & $\textcolor{db}{\mathbf{55.89}}$ & $23.48$ & $77.72$ & $46.89$ \\
\midrule
Nestedformer & MICCAI'22 & $61.38$ & $265.51$ & $61.38$ & $64.29$ & $40.17$ & $72.95$ & $63.22$ & $32.59$ \\
A2FSeg & MICCAI'23 & $60.86$ & $131.48$ & $60.86$ & $\textcolor{dr}{\mathbf{76.10}}$ & $40.90$ & $32.95$ & $77.02$ & $30.57$ \\
H-DenseFormer & MICCAI'23 & $61.50$ & $252.98$ & $61.41$ & $\textcolor{db}{\mathbf{75.76}}$ & $46.79$ & $34.84$ & $\textcolor{dr}{\mathbf{78.33}}$ & $35.31$ \\
\midrule
SAM-Med3D (CT)& TNNLS'25 & $13.13$ & $101.24$ & $19.82$ & $16.70$ & $27.52$ & $\textcolor{db}{\mathbf{18.84}}$ & $43.94$ & $24.46$ \\
SAM-Med3D (PET)& TNNLS'25 & $26.59$ & $101.94$ & $31.92$ & $31.86$ & $31.94$ & $\textcolor{dr}{\mathbf{18.03}}$ & $69.69$ & $24.35$ \\
% DINOv3-L (CT)& Arxiv'25 & $0.00$ & NaN & $0.39$ & $0.00$ & $0.00$ & NaN & $0.00$ & $0.00$ \\
DINOv3-L (PET)& Arxiv'25 & $10.87$ & $-$ & $6.85$ & $64.96$ & $9.43$ & $-$ & $9.40$ & $25.93$ \\
DINOv3-L (CT+PET)& Arxiv'25 & $12.17$ & $-$ & $7.50$ & $71.16$ & $30.86$ & $-$ & $34.99$ & $39.98$ \\
\midrule
SegFormer-3D & CVPRW'24 & $40.44$ & $174.43$ & $56.73$ & $38.19$ & $48.47$ & $54.29$ & $73.63$ & $38.35$ \\
Slim UNETR & TMI'24 & $52.53$ & $310.53$ & $53.99$ & $66.55$ & $49.40$ & $56.55$ & $69.53$ & $41.20$ \\
SuperLightNet & CVPR'25 & $48.35$ & $\textcolor{dr}{\mathbf{59.09}}$ & $60.82$ & $47.61$ & $50.03$ & $34.36$ & $75.29$ & $40.65$ \\
HCMA-UNet & ICME'25 & $43.40$ & $146.11$ & $43.32$ & $62.46$ & $42.06$ & $146.11$ & $67.68$ & $33.18$ \\
U-RWKV & MICCAI'25 & $57.18$ & $\textcolor{db}{\mathbf{61.12}}$ & $\textcolor{db}{\mathbf{66.69}}$ & $59.40$ & $45.97$ & $56.83$ & $64.52$ & $39.71$ \\
\gr VeloxSeg & \gr Ours & \gr $\textcolor{dr}{\mathbf{62.51}}$ & \gr $241.08$ & \gr $\textcolor{dr}{\mathbf{67.76}}$ & \gr $66.28$ & \gr $\textcolor{dr}{\mathbf{56.48}}$ & \gr $47.66$ & \gr $74.81$ & \gr $\textcolor{db}{\mathbf{49.24}}$ \\
\bottomrule
\end{tabular}
\begin{tablenotes}[flushleft]
\footnotesize
\item i) Due to the small object and camouflage recognition involved, DINOv3-L (CT) cannot recognize tumors.
\item ii) ``$-$'' means that the value is out of range.
\end{tablenotes}
\caption{Comparisons of segmentation performance on PET/CT datasets. The best performance is highligted by \textcolor{dr}{\textbf{red}}, followed by \textcolor{db}{\textbf{blue}}. VeloxSeg is highlighted in \colorbox{caption_green}{green}.}
\label{tab:pet/ct_segmentation_performance}
\end{threeparttable}
\vspace{-0.5cm}
\end{table}
Figure~\ref{fig:dice_thr} provides a more intuitive comparison of the trade-offs between Dice and parameter count, and between Dice and GPU throughput. Specifically, regarding segmentation performance, Table~\ref {tab:pet/ct_segmentation_performance} shows the segmentation performance for PET/CT. Appendix~\ref{appendix:qualitative_results} shows the qualitative results of all models. Detailed computational costs are provided in the Appendix~\ref {appendix:details_computational_performance}. Furthermore, to release the model's potential, we train VeloxSeg on the nnUNet~\citep{nnunet} training framework and compare it with the nnUNet baseline, as shown in Appendix~\ref{appendix:nnunet_train_result}. In addition, to verify the modality adaptation ability of the method, we test the performance of MRI segmentation on BraTS2021.
\textbf{Comparison with Basic Models.} Against established basic architectures, including CNN-based, CNN-Transformer-based, and CNN-KAN-based methods, VeloxSeg demonstrates superior performance, with significantly lower computational cost. On the AutoPET-II dataset, VeloxSeg achieves a 62.51\% Dice. This result marginally outperforms the best basic model, VSmTrans, using only 13.30\% of its parameters and 1.96\% of its GFLOPs. On Hecktor2022, VeloxSeg still surpasses all other models. These demonstrate that VeloxSeg is an efficient model in medical segmentation.
\textbf{Comparison with Multimodal Models.} When compared to specialized multimodal architectures, VeloxSeg demonstrates its effectiveness and efficiency in cross-modal feature integration. On the AutoPET-II dataset, VeloxSeg's Dice of 62.51\% outperformed H-DenseFormer, Nestedformer, and A2FSeg by 1.01\%, 1.13\%, and 1.65\%, respectively, while achieving GPU throughput improvements of 2.80$\times$ to 7.75$\times$ and a significant reduction of computational complexity. Furthermore, on Hecktor2022, due to reduced data size, other multimodal models exhibit overfitting and overly conservative predictions, while VeloxSeg's Dice score remains stable.
\textbf{Comparison with Lightweight Models.} Against other lightweight methods, VeloxSeg is clearly superior. It leads in Dice on both datasets by a significant margin of over 5\%. While some competitors have fewer parameters, they are computationally expensive or lack CPU support for clinical use. VeloxSeg offers the best balance, achieving 1.66 MParams and 1.79 GFLOPs. It also achieves a high GPU throughput of 599.06 patches/s and supports CPU-only devices, making it the most clinically practical solution.
\textbf{Train on nnUNet Training Framework.} As analyzed in Appendix~\ref{appendix:nnunet_train_result}, VeloxSeg achieves a 14.2\% Dice improvement with only 1.87\% of nnUNet's MParams and 0.058\% of its GFLOPs~\citep{nnunet}, accompanied by a 4.8$\times$ improvement in GPU throughput and a 52.5$\times$ on CPU.
\textbf{Modality Adaptation Evaluation.} On the BraTS2021 dataset, which contains 4 MRI modalities, our early fusion strategy VeloxSeg-C achieves superior performance, surpassing the second-best method by 1.72\% Dice. This demonstrates that our VeloxSeg can adapt to diverse multimodal segmentation tasks. Details can be found in Appendix~\ref{appendix:modality_adaptation_evaluation}.
\subsection{Module Ablation}
\begin{table}[t]
\centering
\small
\setlength{\tabcolsep}{3pt}
\renewcommand{\arraystretch}{1}
\begin{tabular}{ccc cl ccc c}
\toprule
\multicolumn{3}{c}{\textbf{Modules}} & \multirow{2}{*}{\textbf{Ablation}} & \multirow{2}{*}{\textbf{Hyper-Parameters}} & \multirow{2}{*}{\textbf{Params}} & \multirow{2}{*}{\textbf{FLOPs}} & \multirow{2}{*}{\textbf{Thr. GPU}} & \multirow{2}{*}{\textbf{Dice}} \\
\cmidrule(lr){1-3}
\textbf{Conv.} & \textbf{Trans.} & \textbf{SDKT.} & & & \textbf{(M)} & \textbf{(G)} & \textbf{(Pat./s)} & \textbf{(\%)} \\
\midrule
\checkmark & $\times$ & $\times$ & \multirow{2}{*}{Width} & $\langle 32,64, 128,256\rangle$ & 2.65 & 5.31 & 322.33 & 48.96\\
\checkmark & $\times$ & $\times$ & & \gr$\langle 16,32,64,128\rangle$ & \gr\textbf{\textcolor{db}{0.73}} & \gr2.41 & \gr616.53 & \gr50.10 \\
\cmidrule(lr){4-9}
\checkmark & $\times$ & $\times$ & \multirow{2}{*}{Kernel Size} & $\langle 7\rangle$ & 0.73 & 2.41 & 616.53 & 50.10\\
\checkmark & $\times$ & $\times$ & & \gr$\langle 1,3,5\rangle$ & \gr\textbf{\textcolor{dr}{0.66}} & \gr\textbf{\textcolor{db}{2.30}} & \gr\textbf{\textcolor{db}{641.17}} & \gr53.65 \\
\cmidrule(lr){4-9}
\checkmark & $\times$ & $\times$ & \multirow{6}{*}{Group Size} & $\langle 1,1,1,1\rangle$ & 0.66 & 2.30 & 641.17 & 53.65 \\
\checkmark & $\times$ & $\times$ & & $\langle 1,2,2,4\rangle$ & 0.75 & 2.33 & 620.08 & 53.95 \\
\checkmark & $\times$ & $\times$ & & $\langle 2,4,4,8\rangle$ & 0.89 & 2.44 & 567.79 & 54.40 \\
\checkmark & $\times$ & $\times$ & & \gr$\langle 4,8,8,16\rangle$ & \gr1.18 & \gr2.66 & \gr570.54 & \gr55.84 \\
\checkmark & $\times$ & $\times$ & & $\langle 8,16,16,32\rangle$ & 1.75 & 3.11 & 585.21 & 55.14 \\
\checkmark & $\times$ & $\times$ & & $\langle 16,32,64,128\rangle$ & 4.76 & 4.18 & \textbf{\textcolor{dr}{645.47}} & 56.20 \\
\cmidrule(lr){4-9}
\checkmark & \checkmark & $\times$ & \multirow{2}{*}{Attention Depth} & $\langle 2,2,2,2\rangle$ & 2.37 & 3.07 & 238.54 & 59.56 \\
\checkmark & \checkmark & $\times$ & & \gr$\langle 1,1,1,1\rangle$ & \gr1.88 & \gr2.90 & \gr331.56 & \gr61.03 \\
\cmidrule(lr){4-9}
\checkmark & \checkmark & $\times$ & \multirow{2}{*}{Expansion Ratio} & $\langle 4,4,4,4\rangle$ & 1.88 & 2.90 & 331.56 & 61.03 \\
\checkmark & \checkmark & $\times$ & & \gr$\langle 3,3,2,2\rangle$ & \gr1.61 & \gr2.84 & \gr336.94 & \gr\textbf{\textcolor{db}{61.43}} \\
\cmidrule(lr){4-9}
\checkmark & \checkmark & \checkmark & Teacher & + Texture Teacher & 1.61 & 2.84 & 336.94 & 59.64 \\
\cmidrule(lr){4-9}
\checkmark & \checkmark & \checkmark & Up & Unify Upsampling & 1.66 & 1.79 & 599.06 & 59.71 \\
\cmidrule(lr){4-9}
\checkmark & \checkmark & \checkmark & Gram & + Gram Supervision & 1.66 & \textbf{\textcolor{dr}{1.79}} & 599.06 & \textbf{\textcolor{dr}{62.51}} \\
\bottomrule
\end{tabular}
\caption{Module ablation experiments on AutoPET-II. ``Conv.'': convolution encoder; ``Trans.'': transformer encoder; ``SDKT.'': spatially decoupled knowledge transfer. The best performance is in \textcolor{dr}{\textbf{red}} and the second is in \textcolor{db}{\textbf{blue}}. Final setting is highlighted in \colorbox{caption_green}{green}.}
\label{tab:module_ablation}
\end{table}
We evaluate the performance of three model designs on AutoPET-II: JLC, PWA, and SDKT (Table~\ref{tab:module_ablation}). When using JLC alone, Params and Dice have the lowest performance. Although the framework is the simplest, the FLOPs/throughput is suboptimal due to the use of transposed convolution for upsampling. After adding the attention mechanism, the accuracy increased by 5.59\%, but the throughput decreased by 233.6 Patches/s. After changing the upsampling strategy, FLOPs are significantly reduced from 2.84 G to 1.79 G, and the GPU throughput is increased from 336.94 to 599.06 Patches/s. The last three rows in the table show that it is not enough to just optimize the encoder’s detail representation after adding the texture teacher. Only through the SDKT strategy based on Gram matrices can the representation ability be improved. For more specific reasons and analysis of hyper-parameter selection, please see Appendix~\ref{appendix:hyperparameter_analysis}.
\begin{figure}[t]
\centering
% Left figure: Attention Distance
\begin{minipage}[t]{0.48\textwidth}
\centering
\includegraphics[width=\textwidth]{figure/Attention_Distance.pdf}
\caption{Average attention distance distribution of PWA on AutoPET-II. (a)-(d) show the results for PWA across 4 stage. Y-axis: average attention distance; X-axis: big window size. Wilcoxon rank-sum test: ns ($0.05 < p \le 1$), * ($0.01 < p \le 0.05$), **** ($p \le 0.0001$).}
\label{fig:attention_distance}
\end{minipage}
\hfill
% Right figure: t-SNE and Feature Visualization
\begin{minipage}[t]{0.48\textwidth}
\centering
\includegraphics[width=\textwidth]{figure/TSNE_Feature.pdf}
\caption{Visualization of model decoding and t-SNE plot. (a)-(b) show results without attention; (c)-(d) with attention. In (a) and (c), ``Low Met.'' and ``High Met.'' represent low/high-metabolism PET regions. In (b) and (d), CT background with red tumor outline.}
\label{fig:tsne_feature_visualization}
\end{minipage}
\end{figure}
\begin{table}[t]
\centering
\begin{minipage}[b]{0.4\textwidth}
\centering
\small
\setlength{\tabcolsep}{3pt}
\renewcommand{\arraystretch}{1.2}
\begin{tabular}{lcc}
\toprule
\textbf{Methods} & \textbf{MParams} $\downarrow$ & \textbf{Dice} $\uparrow$ \\
\midrule
FC. & 4.78 \textcolor{db}{+3.58} & 77.15 \textcolor{dr}{+0.12} \\
FC. w. Pn. & 1.27 \textcolor{db}{+0.07} & 59.81 \textcolor{db}{-17.22} \\
\gr JLC & \gr1.20 & \gr77.03 \\
\bottomrule
\end{tabular}
\captionof{table}{Domain generalization capability comparison of the JLC and $\ell_2$ pruning methods~\citep{l2_pruning, torch_pruning} (BraTS2021 $\rightarrow$ MSD2019). ``FC.'' represents full convolution, and ``Pn.'' represents pruning operation.}
\label{tab:jl_conv_vs_prune}
\end{minipage}
\hfill
\begin{minipage}[b]{0.56\textwidth}
\centering
\includegraphics[width=\textwidth]{figure/dice_groupsize.pdf}
\captionof{figure}{Dice performance comparison between different group size configurations.}
\label{fig:dice_groupsize}
\end{minipage}
\end{table}
\textbf{PWA Effect Evaluation.} VolexSeg utilizes PWA to parallelize the computation of multi-scale relationships and reduce redundancy. Experimental details of Figure ~\ref{fig:attention_distance} can be found in Appendix~\ref{appendix:computation_mean_attention_distance}. The inter-group differences in PWA are significant and positively correlated with the window size, indicating that redundant information is reduced and long-distance modeling is efficient. Regarding the fourth attention stage, its design is more similar to multi-head attention, retaining some redundancy. The visualization of the model’s decoding and its t-SNE projection in Figure~\ref{fig:tsne_feature_visualization} demonstrates that PWA helps distinguish tumor regions from high-metabolic regions, while yielding a more compact feature distribution. To verify the effectiveness of PWA in heterogeneous modal modeling, we test various modal input combinations; details are provided in Appendix~\ref{appendix:pwa_multimodal_evaluation}. Notably, introducing modal interaction into PWA improves the Dice score by 5.75\%, significantly enhancing performance robustness without significantly increasing computational costs.
\textbf{JLC Robustness Evaluation.} To verify the robustness of JLC, we conduct three experiments: 1) Comparison of segmentation performance between the JL setting and the standard setting with similar parameter sizes, as shown in Figure~\ref{fig:dice_groupsize}; 2) Comparison of external test performance between the JL setting and the pruned setting with similar parameter sizes, as shown in Table~\ref{tab:jl_conv_vs_prune}; 3) Testing the segmentation performance of JLC on two other datasets, as detailed in the Appendix~\ref{appendix:jl_comparison_hecktor_brast}.
In the first experiment, we use a uniform kernel size of $3$ to ensure a general setup. The JL-guided configuration, $\left\{n,2n,2n,4n\right\}$, consistently surpasses the larger setup, $\left\{2n,2n,2n,2n\right\}$, in all cases. The best performance is a 54.14\% Dice score, achieved when $n=4$. This suggests the JL-guided group size arrangement enables more robust feature extraction in a lightweight model. Table~\ref{tab:jl_conv_vs_prune} verifies the generalization advantage of our lightweight convolution over the pruning method. The model with $\ell_2$ pruning~\citep{l2_pruning} on full convolution performs significantly worse than JLC on MSD2019, even after a cycle of training, pruning, and retraining. As Appendix~\ref{appendix:jl_comparison_hecktor_brast} shown, we test the segmentation performance of pure convolutional networks. The convolution with JL-guided group sizes consistently outperforms the depth-wise convolution, achieving performance gains of 6.25\% on Hecktor2022 and 1.16\% on BraTS2021, with only a marginal increase of 0.091 million parameters. Notably, on the Hecktor2022 dataset, the JLC even surpasses the segmentation performance of the full convolution while using 0.63 million fewer parameters.
\begin{table}[t]
\centering
\begin{minipage}[b]{0.4\textwidth}
\centering
\small
\setlength{\tabcolsep}{3pt}
\renewcommand{\arraystretch}{1.2}
\begin{tabular}{lll}
\toprule
\textbf{Method} & \textbf{Strategy} & \textbf{Dice} $\uparrow$ \\
\midrule
\multirow{5}{*}{Conv.$+$Trans.}
& $-$ & 59.71 \\
& $+\ell_1$ & 1.67 \textcolor{db}{-58.04} \\
& $+$Affinity & 41.44 \textcolor{db}{-18.27} \\
& $+$Shared ROI & 57.15 \textcolor{db}{-2.56} \\
& \gr $+$SDKT & \gr 62.51 \textcolor{dr}{+2.80}\\
\bottomrule
\end{tabular}
\captionof{table}{Comparison of knowledge transfer paths constructed with different losses: \citet{cogseg} use $\ell_1$ loss, \citet{pfseg} use Affinity loss, \citet{ds2f} use agent loss in shared ROI, and we use SDKT.}
\label{tab:dice_multitask_loss}
\end{minipage}
\hfill
\begin{minipage}[b]{0.56\textwidth}
\centering
\includegraphics[width=\textwidth]{figure/Dice_TumorSize.pdf}
\captionof{figure}{Box plots of Dices at different lesion volumes. ``MM.'': PWA$+$JLC multimodal backbone; ``MM.$+$RC.'': backbone with reconstruction teacher; ``Ours'': backbone with the teacher and SDKT.}
\label{fig:dice_tumorsize}
\end{minipage}
\vspace{-0.5cm}
\end{table}
\textbf{Gram-Based Transfer Effect Evaluation.} Our method is the only one to demonstrate positive knowledge transfer, as shown in Table~\ref{tab:dice_multitask_loss}. This is due to our method's avoidance of irrelevant features from the texture teachers, leading to better convergence. To further evaluate the effectiveness of SDKT, we analyze Dice across varying lesion volumes, as shown in Figure~\ref{fig:dice_tumorsize}. VeloxSeg outperforms other lightweight models in segmentation mask fineness across all lesion volumes. Notably, for lesion volumes between 1000 and 5000, ``MM.'', ``MM.+RC.'', and ``Ours'' show a significant upward trend. We attribute this to the increased influence of complex textures in tumor segmentation at these sizes. The above experiments show that there is potential for optimization in small lesion segmentation. The loss weight hyperparameter experiment can be found in Appendix~\ref{appendix:hyperparameter_analysis}.
\section{Conclusion}
In this paper, we propose VeloxSeg, a lightweight, theory-based framework that systematically alleviates the ``\textit{efficiency / robustness conflict}'' in 3D medical image segmentation. By extending the Johnson-Lindenstrauss lemma to the convolution setting, we derive a theoretical lower bound on the group size of convolution per stage, ensuring spatial adjacency and enabling robust detail extraction. Our paired window attention mechanism, by ensembling a tumor localization team composed of attention at different scales, has near-linear complexity and more powerful modeling capabilities. Furthermore, the multimodal interaction of PWA significantly enhances model representation. Furthermore, our Spatially Decoupled Knowledge Transfer strategy establishes a positive knowledge transfer path between the self-supervised texture teacher and the segmentation network, enabling detailed representations that surpass baseline models without increasing inference overhead. Comprehensive evaluation on four diverse clinical datasets demonstrates that VeloxSeg achieves strong robustness with minimal computational cost, requiring only a single CPU core.
\newpage
\bibliography{iclr2026_conference}
\bibliographystyle{iclr2026_conference}
\newpage
\appendix
\section*{Appendix}
\label{appendix}
\paragraph{Table of content:}
\begin{itemize}
\item \S\ref{appendix:reproducibility_statement}: Reproducibility Statement
\item \S\ref{appendix:use_of_LLM}: Use of Large Language Models (LLMs)
\item \S\ref{appendix:characteristics_medical_multi_modality}: Necessity of Multi-Scale Attention
\item \S\ref{appendix:details_PWA}: Details of PWA
\item \S\ref{appendix:detials_jl_group_size}: Details of JL-guided Group Size
\item \S\ref{appendix:dataset_details}: Dataset Details
\item \S\ref{appendix:details_computational_performance}: Details of computational performance
\item \S\ref{appendix:nnunet_train_result}: Results on the nnUNet training framework
\item \S\ref{appendix:qualitative_results}: Qualitative Results
\item \S\ref{appendix:modality_adaptation_evaluation}: Modality Adaptation Evaluation
\item \S\ref{appendix:hyperparameter_analysis}: Hyperparameter Analysis
\item \S\ref{appendix:computation_mean_attention_distance}: Computation of Mean Attention Distance
\item \S\ref{appendix:pwa_multimodal_evaluation}: PWA MultiModal Evaluation
\item \S\ref{appendix:jl_comparison_hecktor_brast}: JL-Setting Generalization Evaluation
\end{itemize}
\section{Reproducibility Statement}
\label{appendix:reproducibility_statement}
Our code is available for download at the following anonymous link: \url{https://github.com/JinPLu/VeloxSeg}. We also provide the source code in the code folder in the supplementary materials. The ``README.md'' file in the source code fully explains the entire training process, including data preprocessing, training, and testing code.
\section{Use of Large Language Models (LLMs)}
\label{appendix:use_of_LLM}
To enhance the quality and readability of this manuscript, we use Large Language Models (LLMs) for assistance with the following tasks:
\begin{enumerate}
\item \textbf{Table Formatting}: Improving the presentation of tables, including adjustments to spacing, typography, and alignment to conform to publication standards.
\item \textbf{Proofreading}: Identifying and correcting grammatical errors, such as improper tense and word usage.
\item \textbf{Language Refinement}: Refining phrasing and sentence structure to improve clarity, conciseness, and overall flow.
\end{enumerate}
\section{Necessity of Multi-Scale Attention}
\label{appendix:characteristics_medical_multi_modality}
CT scans the human body using X-rays and reconstructs a two-dimensional image from one-dimensional projection data. These two-dimensional images are then stacked into a continuous three-dimensional image. CT imaging is characterized by high resolution, low tumor specificity, and rich structural information~\citep{ct_therapy}.
PET generally refers to 18F-FDG PET. Radiologists use the short-lived radionuclide 18F to label glucose. After injecting this labeled glucose into the body, they observe the accumulation of glucose, which indirectly reflects the metabolic activity of human tissues. Because tumors require large amounts of glucose to support their growth and proliferation, tumor areas often appear bright in PET images~\citep{pet/ct1, pet/ct2, pet/ct3}.
Magnetic resonance imaging (MRI) provides rich complementary information for analyzing brain tumors and is routinely used in clinical practice. Specifically, for gliomas, commonly used MRI sequences include T1-weighted (T1), contrast-enhanced T1-weighted (T1Gd), T2-weighted (T2), and T2 fluid-attenuated inversion recovery (T2-FLAIR) images; each sequence plays a different role in distinguishing between the tumor, peritumoral edema, and the tumor core. For meningiomas, these sequences exhibit distinct characteristic features on T1Gd and contrast-enhanced T2-FLAIR (FLAIR-C) MRI images~\citep{mri1,mri_therapy1, mri_therapy2, mri2}.
This indicates that different medical modalities exhibit significant differences in their regions of interest. In tumor imaging, PET imaging, characterized by high metabolic sensitivity and low resolution, excels at localizing tumors at large scales, but its low resolution prevents clear delineation of tumor morphology. While CT imaging is less sensitive for tumors, it excels at clearly delineating tumor tissue contours at small scales. Furthermore, the four contrast types in MRI contribute differently to the identification of targets at three different scales: tumor, peritumoral edema, and tumor core. Therefore, multi-scale modality interaction is crucial in multimodal medical tasks.
\section{Details of PWA}
\label{appendix:details_PWA}
\subsection{Pytorch Code}
\begin{algorithm}[!h]
\SetAlgoLined
\caption{Pytorch Code of Paired Window Attention (PWA)}
\label{algorithm:pwa}
\DontPrintSemicolon
\KwIn{
\begin{itemize}
\item[-] $\mathbf{E}$: Input tensor of shape $[M, C, H, W, D]$.
\item[-] $B$: Min big window size, $[h_b, w_b, d_b]$.
\item[-] $S$: Min small window size, $[h_s, w_s, d_s]$.
\item[-] $r$: Expansion ratio for the paired windows.
\item[-] $\hat{C}$: Number of channels per window after linear projection.
\end{itemize}
}
\KwOut{Attentions of all modalities $\mathbf{A}$}
\SetKwFunction{FDef}{PWA}
\SetKwProg{Fn}{def}{:}{}
\Fn{\FDef{E, B, S, r}}{
$N_{win} \leftarrow \lfloor \log(H / h_b) / \log(r) \rfloor + 1$\tcp*[l]{the number of paired windows}
\;
\tcc{1) Linear Projection}
$\mathbf{Q}, \mathbf{K}, \mathbf{V} \leftarrow \left[\func{PWC}(\func{LN}(\mathbf{E})) \text{ for } \_ \text{ in range(3)}\right]$\;
\;
\tcc{2) Paired Window Gathering}
\For{$\mathbf{X}$ in $[\mathbf{Q}, \mathbf{K}, \mathbf{V}]$}{
$\mathbf{X}s \leftarrow \left[\right]$ \tcp{Initialize list for window features}
\For{$i \leftarrow 1$ \KwTo $N_{win}$}{
\tcp*[l]{Split channels for each window feature}
$\mathbf{X}_i \leftarrow \mathbf{X}[:, (i-1) \cdot \hat{C} : i \cdot \hat{C}, \dots]$\;
\tcp*[l]{Big and small window sizes expand synchronously}
$S_i, B_i \leftarrow r^{i-1} \cdot S, r^{i-1} \cdot B$\;
\tcp{Partition with big window}
$\mathbf{X}_i \leftarrow \func{rearrange}\left(\mathbf{X}_i, \parbox[t]{.5\linewidth}{\texttt{"M Chat (Nh hb) (Nw wb) (Nd db) -> M (Nh Nw Nd Chat) hb wb db"}}\right)$\;
\tcp*[l]{Collecting a salient token for each small window}
$\mathbf{X}_i \leftarrow \func{F.max\_pool3d}(\mathbf{X}_i, S_i, S_i)$ \;
\tcp*[l]{Flatten spatial dims and concatenate multimodal sequences}
$\mathbf{X}_i \leftarrow \func{rearrange}\left(\mathbf{X}_i, \parbox[t]{.35\linewidth}{\texttt{"M (N Chat) nh nw nd -> N Chat (M nh nw nd)"}}\right)$\;
$\mathbf{X}s.\func{append}(\mathbf{X}_i)$\;
}
\tcp{Concatenate all window features}
\tcp*[l]{$\mathbf{X}$: $\left[\sum_{i=1}^{N_{win}}{N_i},\hat{C},M\cdot L\right]$}
$\mathbf{X} \leftarrow \func{torch.cat}(\mathbf{X}s, \text{dim}=0)$ \;
}
\;
\tcc{3) Multimodal Grouped Attention $\times 1$}
$\mathbf{A} \leftarrow \func{multihead\_attention}(\mathbf{Q}, \mathbf{K}, \mathbf{V})$\;
\;
\tcc{4) Paired Window Scattering}
\tcp{Inverse of gathering.}
\tcp{$\mathbf{A}$: $\left[M, \hat{C}, H, W, D\right]$}
$\mathbf{A} \leftarrow \func{window\_scattering}(\mathbf{A})$\;
\;
\tcc{5) Paired Window Mixer}
\tcp{$\mathbf{A}$: $\left[M, C, H, W, D\right]$}
$\mathbf{A} \leftarrow \mathbf{E} + \func{Dropout}(\func{PWC}(\mathbf{A}))$\;
\;
\KwRet{A}\;
}
\end{algorithm}
We've organized the PyTorch code and feature shape changes of PWA to help readers understand its key operations. As shown in the Algorithm~\ref{algorithm:pwa}, $N_{win} = \log(H / h_b) / \log(r) + 1$, which means that we expand the large window $\left(h_b, w_b, d_b\right)$ by $N_{win}-1$ to obtain full-image-sized features. In the AutoPETII dataset, we set the minimum large window size of each stage to $\left\langle3,3,3\right\rangle,\left\langle6,6,6\right\rangle,\left\langle3,3,3\right\rangle,\left\langle3,3,3\right\rangle$, which means that after the synchronous expansion of the paired windows, the large window sizes of each stage are:
\begin{itemize}
\item First Stage: $\left\langle3,3,3\right\rangle,\left\langle6,6,6\right\rangle,\left\langle12,12,12\right\rangle,\left\langle24,24,24\right\rangle$;
\item Second Stage: $\left\langle6,6,6\right\rangle,\left\langle12,12,12\right\rangle$;
\item Third Stage: $\left\langle3,3,3\right\rangle,\left\langle6,6,6\right\rangle$;
\item Forth Stage: $\left\langle3,3,3\right\rangle$.
\end{itemize}
The settings of BraTS2021 are the same. In the Hecktor2022 dataset, the minimum maximum window size at each stage is $\left\langle4,4,2\right\rangle,\left\langle8,8,4\right\rangle,\left\langle4,4,2\right\rangle,\left\langle4,4,2\right\rangle$. The number of windows must be divisible by the number of channels of the feature map at the current stage to avoid extensive output channels during linear mapping. Therefore, the minimum maximum window size in the second stage is doubled.
\subsection{Feature Flow}
\begin{figure}[h]
\centering
\includegraphics[width=\textwidth]{figure/PWA.pdf}
\caption{Detailed architecture of Paired Window Attention (PWA). This figure focuses on visually showing the feature flows of PWA.}
\label{fig:detail_pwa}
\end{figure}
As shown in Figure~\ref{fig:detail_pwa}, given the $m$-th modal feature of the $k$-th encoder stage, $\mathbf{E}_m^k\in \mathbb{R} ^{C^k\times H^k\times W^k\times D^k}$, we need to first compute a set of ordered paired window sizes $\left\{ Win_{i}^{k} \right\} _{i=1}^{N_{win}^k}$, where $N_{win}^k$ is the number of window pairs:
\begin{equation}
\left\{ Win_{i}^{k} \right\} _{i=1}^{N_{win}^{k}}=\left\{ B_{i}^{k},S_{i}^{k} \right\} _{i=1}^{N_{win}^{k}}=\left\{ \begin{array}{c}
\left( r^{i-1}h_{b}^{k},r^{i-1}w_{b}^{k},r^{i-1}d_{b}^{k} \right) ,\\
\left( r^{i-1}h_{s}^{k},r^{i-1}w_{s}^{k},r^{i-1}d_{s}^{k} \right)\\
\end{array} \right\} _{i=1}^{N_{win}^{k}},
\end{equation}
where $r\in \mathbb{N}$ is the expansion rate (default $r=2$), $B_i^k$ and $S_i^k$ represent the big window and small window, respectively. $h_{b}^{k}$, $w_{b}^{k}$, $d_{b}^{k}$ represent the height, width, and depth of the big window; $h_{s}^{k}$, $w_{s}^{k}$, $d_{s}^{k}$ correspond to the small window. Notably, $r$, $B_1^k$, and $S_1^k$ are closely related to the computational cost, and the specific settings for different datasets are given in Appendix~\ref{appendix:attention_computational_complexity_analysis}. Since $B_{N_{win}^k}^k$ is equal to $\left(H^k,W^k,D^k \right)$, there is no need to set $N_{win}^k$ in advance. Subsequently, the encoder features will undergo the following steps in order:
\subsubsection{Linear Projection}
We do not follow the habit of linear mapping: the number of output channels is equal to the number of input channels, but based on the JL-guided minimum head size $C_{min}^{k}$, the number of heads $N_{head}^k$, and the number of window pairs $N_{win}^k$, the number of output channels is $\hat{C}^k=\min \left\{ nC_{min}^{k}, n\in \mathbb{N} : N_{win}^kN_{head}^k \left(nC_{min}^{k}\right)\ge C^k \right\}$, where $\hat{C}^k$ is the actual head size. The formula is as follows:
\begin{equation}
\mathbf{Q}^k_m=\text{PWC}\left( \text{LN}\left( \mathbf{E}_{m}^{k} \right) \right),
\mathbf{K}^k_m=\text{PWC}\left( \text{LN}\left( \mathbf{E}_{m}^{k} \right) \right),
\mathbf{V}^k_m=\text{PWC}\left( \text{LN}\left( \mathbf{E}_{m}^{k} \right) \right),
\end{equation}
where $\mathbf{Q},\mathbf{K},\mathbf{V}$ represent query, key, and value respectively. $\text{LN}\left(\cdot\right)$ represents layer normalization, and $\text{PWC}\left(\cdot\right)$ represents point-wise convolution. For convenience, we use $\mathbf{X}$ to represent $\mathbf{Q}$, $\mathbf{K}$, $\mathbf{V}$ in Algorithm~\ref{algorithm:pwa} and Figure~\ref{fig:method} (a).
\subsubsection{Paired Window Gathering}
Synchronously expanding paired windows ensures that the sequence lengths of $\mathbf{Q}$, $\mathbf{K}$ and $\mathbf{V}$ remain consistent across multiple scales, which enables parallel computation. Given $\mathbf{Q}_{m}^k, \mathbf{K}_{m}^k, \mathbf{V}_{m}^k \in \mathbb{R}^{C^k \times H^k \times W^k \times D^k}$, where $i$ denotes the $i$-th paired window, the processing pipeline is as follows:
\begin{itemize}
\item \textbf{Channel Split}: This operation assigns features to their corresponding windows along the channel dimension. The feature shape becomes $\left(N_{\text{head}}^k \times \hat{C}^k \times H^k \times W^k \times D^k\right)$.
\item \textbf{Big Window Partition}: This operation partitions the features into non-overlapping blocks based on the big window size $B_i^k$. The feature shape becomes $\left( n^k_i, N_{\text{head}}^k, \hat{C}^k, r^{i-1}h_b, r^{i-1}w_b, r^{i-1}d_b \right)$, where $n^k_i$ is the total number of large windows.
\item \textbf{Small Window Pooling}: This operation gathers salient tokens from each small window $S_i^k$. The feature shape becomes $\left(n^k_i, N_{\text{head}}^k, \hat{C}^k, h_{b}^{k}/h_{s}^{k}, w_{b}^{k}/w_{s}^{k}, d_{b}^{k}/d_{s}^{k}\right)$.
\item \textbf{Flatten and Concatenation}: These are feature reshaping operations. The new feature size is $\left(n^k_i, N_{\text{head}}^k, \hat{C}^k, ML \right)$, where $L = (h_{b}^{k}/h_{s}^{k}) \times (w_{b}^{k}/w_{s}^{k}) \times (d_{b}^{k}/d_{s}^{k})$ is the sequence length and $M$ is the number of modalities.
\end{itemize}
The above operations are repeated for each paired window. Thanks to the synchronous expansion, the sequence length $L$ is guaranteed to be equal across different scales. Finally, we can summarize this process as follows:
\begin{equation}
\mathbf{\tilde{Q}}^{k}=\text{Gather}\left( \mathbf{Q}_{1}^{k},\cdots,\mathbf{Q}_{M}^{k} \right),
\mathbf{\tilde{K}}^{k}=\text{Gather}\left( \mathbf{K}_{1}^{k},\cdots,\mathbf{K}_{M}^{k} \right),
\mathbf{\tilde{V}}^{k}=\text{Gather}\left( \mathbf{V}_{1}^{k},\cdots,\mathbf{V}_{M}^{k} \right),
\end{equation}
\subsubsection{Multimodal Grouped Attention}
The similarity matrix calculation formula is as follows:
\begin{equation}
\mathbf{S}^k=\frac{1}{\sqrt{\hat{C}^k}}\left(\tilde{\mathbf{Q}}^k\right)^T\otimes \mathbf{\tilde{K}}^k,
\end{equation}
where $\otimes$ represents matrix multiplication. In addition, each $L\times L$ block in the similarity matrix is assigned a relative position code $\mathbf{E}_{pos}^k$ to strengthen the position relationship between voxels in the window and cross-modal voxels. The remaining attention is calculated as follows:
\begin{equation}
\mathbf{W}^k=\text{softmax}\left(\mathbf{S}^k+\mathbf{E}_{pos}^{k} \right) ,
\end{equation}
\begin{equation}
\mathbf{A}^k=\mathbf{W}^k\otimes \mathbf{\hat{V}}^k,
\end{equation}
$\mathbf{W}^k$ is the attention weight matrix, and $\mathbf{A}^k$ is the attention obtained for each window.
\subsubsection{Paired Window Scattering}
After computing the attention mechanism in parallel, we perform the inverse operation of Paired Window Gathering to map the multi-scale attention mechanism to the original feature space, obtaining the window attention $\mathbf{A}_{m}^k$ for each modality, which has the same size as $\mathbf{Q}_{m}^k, \mathbf{K}_{m}^k, \mathbf{V}_{m}^k$.
\begin{equation}
\mathbf{A}_{1}^k,\cdots, \mathbf{A}_{M}^k = \text{Scatter}(\mathbf{A}^k).
\end{equation}
\subsubsection{Paired Window Mixer}
The above operations obtain window attention of different scales. We will use $1\times 1\times 1$ convolution to mix them to get the final feature $\mathbf{\tilde{E}}_{m}^{k}$. The formula is as follows:
\begin{equation}
\mathbf{\tilde{E}}_{m}^{k} = \mathbf{E}_{m}^{k}+\text{PWC}\left(\mathbf{A}_m^k\right), m=1,\cdots,M.
\end{equation}
\subsection{Computational Complexity}
\label{appendix:attention_computational_complexity_analysis}
% \begin{table}[h]
% \centering
% \small
% \begin{tabular}{@{}ll@{}}
% \toprule
% \textbf{Method} & \textbf{Complexity} \\
% \midrule
% Swin Transformer & $\mathcal{O}(N(4C^2 + 2BC))$ \\
% CSin Transformer & $\mathcal{O}(N(4C^2 + 2(H{+}W{+}D) C))$ \\
% SegFormer & $\mathcal{O}(4NC^2 + 2(N^2/v) C)$ \\
% UNETR++ & $\mathcal{O}(4NC^2 + 2(N^2/v) C + 2Nk^3 C)$ \\
% Slim UNETR & $\mathcal{O}(4NC^2 + 2(N^2/v) C + 2Nk^3 C)$ \\
% \gr PWA (Ours) & \gr $\mathcal{O}\left(\frac{N\kappa}{S}(C^2 + \frac{B}{S} C)\right)$ \\
% \bottomrule
% \end{tabular}
% \caption{Attention complexity comparisons. $N$ represents volume.}
% \label{tab:attention_complexity}
% \end{table}
Let $N=H\cdot W\cdot D$, $B=h_b\cdot w_b\cdot d_b$, $S=h_s\cdot w_s\cdot d_s$, and $\kappa =1+\frac{1}{r^2}+\cdots +\frac{1}{r^{2N_{win}}}=\frac{1-r^{-3N_{win}}}{1-r^{-3}}$, the computational complexity of PWA is calculated as follows:
\begin{align*}
& \underset{\mathrm{number\ of\ big\ windows}}{\underbrace{\left( \frac{N}{B} \right) \left( 1+\frac{1}{r^2}+\cdots +\frac{1}{r^{2N_{win}}} \right) }}\underset{\mathrm{multiplication\ operations\ per\ big\ window}}{\underbrace{\left( 4\frac{B}{S}C^2+2\frac{B^2}{S^2}C \right) }}
\\
& =N\kappa \left( 4\frac{1}{S}C^2+2\frac{B}{S^2}C \right) \\
& =\left( \frac{N\kappa}{S} \right) \left( 4C^2+2\frac{B}{S}C \right),
\end{align*}
\section{Details of JL-guided Group Size}
\label{appendix:detials_jl_group_size}
For a typical lightweight 3D medical segmentation method, its network contains $M\in\{1,2,4\}$ modalities with volume ratios $\left\{v^k\right\}_{k=1}^4 = \{4^3, 8^3, 16^3, 32^3\}$ of each stage. Its complexity increases with depth, which means that the group size is:
\begin{equation}
C_{\mathrm{group}} \approx \begin{cases}
\left\{4.2\alpha, 6.2\alpha, 8.3\alpha, 10.4\alpha\right\}, & M=1\\
\left\{4.9\alpha, 6.9\alpha, 9.0\alpha, 11.1\alpha\right\}, & M=2\\
\left\{5.5\alpha, 7.6\alpha, 9.7\alpha, 11.8\alpha\right\}, & M=4
\end{cases}.
\end{equation}
Considering that the group size needs to be divisible by the total number of channels and that nonlinear networks have stronger compression capabilities than linear networks, we use $\left\{C_{\mathrm{group}}^k\right\}_{k=1}^4 = \left\{4\alpha, 8\alpha, 8\alpha, 16\alpha\right\}$ for each stage, where $\alpha$ is determined by the most difficult AutoPET-II dataset to ensure universality. For convenience, we replace $\alpha$ with $n=\lceil \alpha/4\rceil\in \mathbb{N}$, and the final convolution group size of each stage of the network is set to $\left\{C_{\mathrm{group}}^k\right\}_{k=1}^4=\left\{n,2n,2n,4n\right\}$.
For lightweight convolution settings in the natural image domain, the input image typically has $M=3$ channels, and the volume ratio of each stage of the network is $\left\{v^k\right\}_{k=1}^4 = \{1^2, 2^2, 4^2, 8^2\} $, which means the group size is: $\left\{C_{\mathrm{group}}^k\right\}_{k=1}^4= \{\alpha\log 3, \alpha\log 12, \alpha\log 48, \alpha\log 192\} \approx \left\{1.1\alpha, 2.5\alpha, 3.9\alpha, 5.3\alpha\right\} $. Considering the integer divisibility of the channels, it is recommended to use a group size of $\{\alpha, 2\alpha, 4\alpha, 4\alpha\} $. This setting is similar to the depth-wise convolution setting, further demonstrating the effectiveness of depth-wise convolution in the natural image domain.
\section{Dataset Details}
\label{appendix:dataset_details}
\begin{table}[h]
\centering
\centering
\small
\setlength{\tabcolsep}{2pt}
\renewcommand{\arraystretch}{1.2}
\begin{tabular}{>{\arraybackslash}m{1.5cm}>{\centering\arraybackslash}m{1.5cm}>{\centering\arraybackslash}m{1cm}>{\arraybackslash}m{3cm}>{\centering\arraybackslash}m{2cm}>{\centering\arraybackslash}m{2cm}>{\centering\arraybackslash}m{2cm}}
\toprule
\centering\textbf{Dataset} & \textbf{Modalities} & \textbf{Region} & \centering\textbf{Label Type} & \textbf{Image Size} & \textbf{Crop Size} & \textbf{Voxel Spacing}\\
\midrule
\rowcolor{gray!10}AutoPET-II & PET, CT & Whole Body & Malignant melanoma, lymphoma, or lung cancer lesions & Min $(400, 400, 200)$ & $(96, 96, 96)$ & Fixed $(2.036, 2.036, 3)$\\
\midrule
Hecktor2022 & PET, CT & Head \& Neck & Primary gross tumor volume (GTVp), or lymph node gross tumor volume (GTVn) & Min $(128, 128, 67)$ & $(128,128,64)$ & Median $(0.98, 0.98, 3.3)$ \\ \midrule
\rowcolor{gray!10}BraTS2021 & MRI & Brain & Brain tumors: whole tumor (WT), tumor core (TC), enhancing tumor (ET) subregions. & Fixed $(240 , 240 , 155)$ & $(96, 96, 96)$ & Fixed $(1.0, 1.0, 1.0)$\\
\midrule
MSD2019 Task-01 & MRI & Brain & Glioma segmentation (multi-class): necrosis/active tumor and edema. & Fixed $(240 , 240 , 155)$ & $(96, 96, 96)$ & Fixed $(1.0, 1.0, 1.0)$ \\
\bottomrule
\end{tabular}
\caption{Details of AutoPET-II, Hecktor2022, BraTS2021, and MSD2019 datasets. If image size is a variable, the minimum value is reported. If voxel spacing is a variable, the median value is reported.}
\label{tab:datasets_summary}
\end{table}
We evaluate our proposed VeloxSeg on four public medical image datasets: AutoPET-II~\citep{AutoPET-II}, Hecktor2022~\citep{Hecktor2022}, BraTS2021~\citep{BraTS2021}, and MSD2019 Task-01~\citep{MSD2019}. The AutoPET-II and Hecktor2022 datasets are multimodal PET/CT datasets for tumor segmentation. AutoPET-II contains 1,014 whole-body PET/CT scans with variable image dimensions and is cropped to $96 \times 96 \times 96$ patches. Hecktor2022 comprises 524 head and neck PET/CT scans, cropped to $128 \times 128 \times 64$ patches. The BraTS2021 and MSD2019 datasets are multimodal (T1, T1ce, T2, FLAIR) MRI datasets for brain tumor segmentation. Each patient's data is registered to a common spatial resolution of $240 \times 240 \times 155$ and undergoes skull stripping. The brain tumor region is segmented into three primary sub-regions: the enhancing tumor (ET), the tumor core (TC), and the whole tumor (WT). For training efficiency, volumes are cropped to $96 \times 96 \times 96$ patches. BraTS2021 contains 1,251 cases for training and validation, while MSD2019 (484 cases) serves as an external test set to evaluate domain generalization capability across different data distributions.
We use four public medical image datasets to verify the effectiveness of SlimMSCT, including AutoPET-II, Hecktor2022, BraTS2021, and MSD2019 Task-01, which is used as an external test set to compare the generalization ability of the model. The first two datasets contain CT and PET images, and the latter two datasets contain 3D MRI images with four modalities. The details of the datasets are described in Table~\ref{tab:datasets_summary}.
\section{Details of computational performance}
\label{appendix:details_computational_performance}
\begin{table}[h]
\centering
\small
\setlength{\tabcolsep}{3pt}
\renewcommand{\arraystretch}{1.2}
\begin{tabular}{l l cccc cccc}
\toprule
\multirow{2}{*}{Methods} & \multirow{2}{*}{Type} & \multicolumn{4}{c}{AutoPET-II} & \multicolumn{4}{c}{Hecktor2022} \\
\cmidrule(lr){3-6}\cmidrule(lr){7-10}
& & MP. & GF. & ThrG. & ThrC. & MP. & GF. & ThrG. & ThrC. \\
\midrule
\multicolumn{10}{l}{\gr\textbf{Basic Models}} \\
UNet & CNN & 5.75 & 136.56 & 101.04 & 0.23 & 5.75 & 161.84 & 85.60 & 0.19 \\
VNet & CNN & 45.60 & 322.22 & 58.99 & 0.14 & 45.60 & 381.89 & 49.82 & 0.11 \\
UNETR & CNN-Transformer & 95.76 & 83.61 & 131.96 & 0.40 & 95.76 & 99.09 & 105.78 & 0.35 \\
Swin UNETR & CNN-Transformer & 15.51 & 84.26 & 38.37 & 0.14 & 15.51 & 100.66 & 28.58 & 0.10 \\
VSmTrans & CNN-Transformer & 12.48 & 91.44 & 36.56 & 0.14 & 3.12 & 28.79 & 34.13 & 0.16 \\
UNETR++ & CNN-Transformer & 19.97 & 57.93 & 161.15 & 0.67 & 19.97 & 68.66 & 138.39 & 0.56 \\
U-KAN & CNN-KAN & 7.06 & 22.90 & 187.06 & 0.82 & 7.06 & 27.13 & 159.92 & 0.75 \\
\midrule
\multicolumn{10}{l}{\gr\textbf{Multimodal Models}} \\
Nestedformer & CNN-Transformer & 4.71 & 58.62 & 95.63 & 0.41 & 4.71 & 69.48 & 79.05 & 0.35 \\
A2FSeg & CNN & 41.32 & 207.97 & 52.02 & 0.17 & 41.32 & 246.48 & 40.60 & 0.13 \\
H-DenseFormer & CNN-Transformer & 3.64 & 71.91 & 123.35 & 0.44 & 3.64 & 85.23 & 102.80 & 0.37 \\
\midrule
\multicolumn{10}{l}{\gr\textbf{Lightweight Models}} \\
SegFormer-3D & CNN-Transformer & 4.50 & 5.11 & \textcolor{db}{\textbf{364.24}} & 3.31 & 4.50 & 6.06 & \textcolor{db}{\textbf{305.01}} & 2.67 \\
Slim UNETR & CNN-Transformer & 1.77 & \textcolor{db}{\textbf{3.83}} & 178.33 & \textcolor{dr}{\textbf{11.40}} & 1.77 & \textcolor{db}{\textbf{4.53}} & 151.85 & \textcolor{dr}{\textbf{8.78}} \\
SuperLightNet & CNN-Transformer & 2.75 & 19.42 & 55.48 & 0.27 & 2.75 & 23.01 & 47.36 & 0.23 \\
HCMA-UNet & CNN-Mamba & 2.81 & 26.15 & 54.51 & $-$ & 2.81 & 31.00 & 46.48 & $-$ \\
U-RWKV & CNN-RWKV & \textcolor{dr}{\textbf{1.44}} & 20.68 & 82.09 & $-$ & \textcolor{dr}{\textbf{1.44}} & 24.51 & 73.98 & $-$ \\
\midrule
\gr VeloxSeg & \gr CNN-Transformer & \gr\textcolor{db}{\textbf{1.66}} & \gr\textcolor{dr}{\textbf{1.79}} & \gr\textcolor{dr}{\textbf{390.91}} & \gr\textcolor{db}{\textbf{6.67}} & \gr\textcolor{db}{\textbf{1.66}} & \gr\textcolor{dr}{\textbf{2.13}} & \gr\textcolor{dr}{\textbf{319.80}} & \gr\textcolor{db}{\textbf{5.47}} \\
\bottomrule
\end{tabular}
\caption{Computational performance comparison of all models on AutoPET-II and Hecktor2022 datasets. ``MP.'': Million Parameters; ``GF.'': GFLOPs; ``ThrG.'': Throughput on GPU; ``ThrC.'': Throughput on CPU.}
\label{tab:autopetii_hecktor2022_computation_performance}
\end{table}
We evaluate the computational performance of VeloxSeg against other leading models on the AutoPET-II, Hecktor2022, and BraTS2021 datasets. Our analysis focus on four key metrics: the number of model parameters in millions, GFLOPs, GPU throughput, and CPU throughput.
On the AutoPET-II and Hecktor2022 datasets, VeloxSeg established a new standard for efficiency. As detailed in Table~\ref{tab:autopetii_hecktor2022_computation_performance}, our model operates with only 1.66 million parameters and the lowest GFLOPs among all competitors, requiring just 1.79 on AutoPET-II and 2.13 on Hecktor2022. This lean profile translates to exceptional speed, where VeloxSeg recorded the highest GPU throughput and second-highest CPU throughput on both datasets. In the lightweight category, while Slim UNETR is marginally smaller, VeloxSeg surpasses it in computational cost and processing speed.
\begin{table}[h]
\centering
\small
\setlength{\tabcolsep}{3pt}
\renewcommand{\arraystretch}{1.2}
\begin{tabular}{l l cccc}
\toprule
\multicolumn{1}{c}{\textbf{Methods}} & \multicolumn{1}{c}{\textbf{Type}} & \multicolumn{1}{c}{\textbf{MP.} $\downarrow$} & \multicolumn{1}{c}{\textbf{GF.} $\downarrow$} & \multicolumn{1}{c}{\textbf{ThrG.} $\uparrow$} & \multicolumn{1}{c}{\textbf{ThrC.} $\uparrow$} \\
\midrule
\multicolumn{6}{l}{\gr\textbf{Basic Models}} \\
UNet & CNN & 5.75 & 138.14 & 103.70 & 0.23 \\
VNet & CNN & 45.61 & 322.85 & 63.37 & 0.14 \\
UNETR & CNN-Transformer & 102.06 & 85.79 & 128.16 & 0.41 \\
Swin UNETR & CNN-Transformer & 15.51 & 85.53 & 38.11 & 0.13 \\
VSmTrans & CNN-Transformer & 12.48 & 92.72 & 36.18 & 0.13 \\
UNETR++ & CNN-Transformer & 19.98 & 58.81 & 153.63 & 0.52 \\
U-KAN & CNN-KAN & 7.06 & 23.69 & 181.89 & 0.85 \\
\midrule
\multicolumn{6}{l}{\gr\textbf{Multimodal Models}} \\
Nestedformer & CNN-Transformer & 7.52 & 88.43 & 56.92 & 0.23 \\
A2FSeg & CNN & 74.55 & 361.18 & 28.92 & 0.08 \\
H-DenseFormer & CNN-Transformer & 5.39 & 73.18 & 95.61 & 0.42 \\
\midrule
\multicolumn{6}{l}{\gr\textbf{Lightweight Models}} \\
SegFormer-3D & CNN-Transformer & 4.53 & \textcolor{db}{\textbf{5.42}} & \textcolor{db}{\textbf{355.73}} & 2.98 \\
Slim UNETR & CNN-Transformer & 1.78 & 6.59 & 97.50 & \textcolor{dr}{\textbf{10.62}} \\
SuperLightNet & CNN-Transformer & 2.75 & 19.54 & 55.13 & 0.28 \\
HCMA-UNet & CNN-Mamba & 2.81 & 26.69 & 53.72 & $-$ \\
U-RWKV & CNN-RWKV & \textcolor{dr}{\textbf{1.43}} & 21.08 & 83.15 & $-$ \\
\midrule
\gr VeloxSeg-C & \gr CNN-Transformer & \gr\textcolor{db}{\textbf{1.46}} & \gr\textcolor{dr}{\textbf{2.64}} & \gr\textcolor{dr}{\textbf{536.62}} & \gr\textcolor{db}{\textbf{5.23}} \\
\bottomrule
\end{tabular}
\caption{Computational performance on BraTS2021 dataset with patch size $96\times96\times96$ and $4$ modalities (T1/T1ce/T2/FLAIR). ``MP.'': Million Parameters; ``GF.'': GFLOPs; ``ThrG.'': Throughput on GPU; ``ThrC.'': Throughput on CPU.}
\label{tab:brats2021_computation_performance}
\end{table}
On the BraTS2021 dataset, we test early-fusion VeloxSeg due to its concentrated target distribution, absence of small lesions, and low modality heterogeneity. Table~\ref{tab:brats2021_computation_performance} shows that VeloxSeg-C is one of the smallest models with only 1.46 million parameters, yet it achieves the lowest GFLOPs at 2.64. Most notably, it delivered the highest GPU throughput of any model, processing 536.62 images/s, alongside the second-fastest CPU throughput. This positions VeloxSeg-C as a more efficient and faster alternative to other lightweight models like U-RWKV and SegFormer-3D.
Across all three benchmarks, the VeloxSeg architecture demonstrates an excellent balance between model size, computational requirements, and processing speed, making it well-suited for deployment in resource-constrained environments. Furthermore, segmentation methods based on sequence models, such as Mamba and RWKV, lack CPU support, significantly limiting their application in edge devices.
\section{Results on the nnUNet training framework}
\label{appendix:nnunet_train_result}
\begin{table}[h]
\centering
\small
\setlength{\tabcolsep}{3pt}
\renewcommand{\arraystretch}{1.2}
\begin{tabular}{l l c c c c c c}
\toprule
\textbf{Dataset} & \textbf{Model} & \textbf{MParams} $\downarrow$ & \textbf{GFLOPs} $\downarrow$ & \textbf{Thr.GPU} $\uparrow$ & \textbf{Thr.CPU} $\uparrow$ & \textbf{Dice} $\uparrow$ & \textbf{HD95} $\downarrow$ \\
\midrule
\multirow{2}{*}{AutoPETII} & nnUNet & 88.62 & 3078.83 & 81.13 & 0.127 & 55.85 & 193.54 \\
& \gr VeloxSeg & \gr\textcolor{dr}{\textbf{1.66}} & \gr\textcolor{dr}{\textbf{1.79}} & \gr\textcolor{dr}{\textbf{390.91}} & \gr\textcolor{dr}{\textbf{6.67}} & \gr\textcolor{dr}{\textbf{70.05}} & \gr\textcolor{dr}{\textbf{177.51}} \\
\midrule
\multirow{2}{*}{Hecktor2022} & nnUNet & 88.62 & 4828.04 & 68.02 & 0.106 & 60.80 & 36.67 \\
& \gr VeloxSeg & \gr\textcolor{dr}{\textbf{1.66}} & \gr\textcolor{dr}{\textbf{2.13}} & \gr\textcolor{dr}{\textbf{319.80}} & \gr\textcolor{dr}{\textbf{5.47}} & \gr\textcolor{dr}{\textbf{62.51}} & \gr\textcolor{dr}{\textbf{30.22}} \\
\bottomrule
\end{tabular}
\caption{Performance comparison between nnUNet and VeloxSeg across PET/CT datasets. Both segmentation performance and computational efficiency are evaluated.}
\label{tab:nnunet_vs_nnveloxseg}
\end{table}
To unleash more model potential, we placed the model in the nnUNet training framework and completed the training while keeping the patch size consistent with the experimental setting. The results are shown in Table~\ref{tab:nnunet_vs_nnveloxseg}. It can be seen that our model has achieved comprehensive transcendence. In the AutoPETII dataset, we achieved a Dice that was 14.2\% higher than the nnUNet baseline with 1.87\% of the parameters and 5.81e-2\% GLOPs, and the GPU throughput and CPU throughput increased by 4.8$\times$ and 52.5$\times$ respectively. Similarly, in the Hecktor2022 dataset, VeloxSeg achieved a Dice that was 1.71\% higher than the nnUNet baseline with 1.87\% of the parameters and 4.41e-2\% GLOPs, and the GPU throughput and CPU throughput increased by 4.7$\times$ and 51.6$\times$ respectively.
\section{Qualitative Results}
\label{appendix:qualitative_results}
\begin{figure*}[h]
\centering
\includegraphics[width=\textwidth]{figure/Pred.pdf}
\caption{3D qualitative visualization of different methods on the AutoPET-II, Hecktor2022 and BraTS2021 datasets. In PET-CT datasets, model predictions are shown on CT images (red indicates true positives, yellow indicates false positives, and blue indicates false negatives); In MRIs datasets, model predictions are shown on T1CE images (red represents ET, yellow represents TC, and green represents WT).}\label{fig:Prediction}
\end{figure*}
Figure~\ref{fig:Prediction} show the qualitative results of three segmentation cases: large melanoma lesions, primary and secondary lesions of right tonsil tumors or lymphomas, and glioma lesions. It can be seen that our method can accurately locate the tumor area, exclude the wrong attention to areas such as intracranial veins, and the prediction results are consistent with the labels.
\section{Modality Adaptation Evaluation}
\label{appendix:modality_adaptation_evaluation}
\begin{table}[h]
\centering
\small
\setlength{\tabcolsep}{3pt}
\renewcommand{\arraystretch}{1.2}
\begin{tabular}{l cccc cccc}
\toprule
\multirow{2}{*}{\textbf{Method}} & \multicolumn{4}{c}{\textbf{Dice Similarity Coefficient (\%)}} & \multicolumn{4}{c}{\textbf{Hausdorff Distance 95\% (mm)}} \\
\cmidrule(lr){2-5}\cmidrule(lr){6-9}
& \multicolumn{1}{c}{\small\textbf{Avg.} $\uparrow$} & \multicolumn{1}{c}{\small\textbf{ET} $\uparrow$} & \multicolumn{1}{c}{\small\textbf{TC} $\uparrow$} & \multicolumn{1}{c}{\small\textbf{WT} $\uparrow$} & \multicolumn{1}{c}{\small\textbf{Avg.} $\downarrow$} & \multicolumn{1}{c}{\small\textbf{ET} $\downarrow$} & \multicolumn{1}{c}{\small\textbf{TC} $\downarrow$} & \multicolumn{1}{c}{\small\textbf{WT} $\downarrow$} \\
\midrule
UNet & $88.18$ & $89.62$ & $85.65$ & $89.28$ & $4.93$ & $5.95$ & $7.52$ & $3.59$ \\
V-Net & $88.86$ & $90.66$ & $86.16$ & $89.75$ & $5.26$ & $5.98$ & $8.29$ & $4.31$ \\
UNETR & $85.44$ & $88.12$ & $81.49$ & $86.71$ & $6.68$ & $8.51$ & $8.19$ & $4.75$ \\
Swin UNETR & $88.52$ & $90.19$ & $85.73$ & $89.63$ & $5.07$ & $6.37$ & $7.29$ & $3.44$ \\
VSmTrans & $86.62$ & $91.15$ & $78.01$ & $90.71$ & $7.00$ & $6.00$ & $12.07$ & $3.44$ \\
UNETR++ & $88.77$ & $90.26$ & $87.01$ & $89.05$ & $4.49$ & $5.30$ & $6.98$ & $3.76$ \\
U-KAN & $88.51$ & $90.57$ & $86.14$ & $88.82$ & $5.44$ & $5.77$ & $8.03$ & $4.65$ \\
\addlinespace[2pt]
Nestedformer & $88.54$ & $89.60$ & $86.71$ & $89.30$ & $\textcolor{db}{\mathbf{4.21}}$ & $5.44$ & $7.38$ & $\textcolor{db}{\mathbf{3.13}}$ \\
A2FSeg & $88.18$ & $\textcolor{db}{\mathbf{91.78}}$ & $84.58$ & $88.18$ & $4.66$ & $\textcolor{db}{\mathbf{4.47}}$ & $7.51$ & $3.72$ \\
H-DenseFormer & $89.35$ & $90.80$ & $86.66$ & $90.59$ & $5.58$ & $5.85$ & $8.88$ & $3.97$ \\
\addlinespace[2pt]
SegFormer-3D & $89.18$ & $90.37$ & $\textcolor{db}{\mathbf{87.49}}$ & $89.69$ & $4.61$ & $5.45$ & $\textcolor{db}{\mathbf{6.30}}$ & $3.57$ \\
Slim UNETR & $87.33$ & $89.31$ & $85.00$ & $87.66$ & $5.16$ & $6.46$ & $7.57$ & $3.52$ \\
SuperLightNet & $\textcolor{db}{\mathbf{89.72}}$ & $91.46$ & $87.22$ & $90.48$ & $4.46$ & $5.39$ & $6.33$ & $\textcolor{dr}{\mathbf{3.06}}$ \\
HCMA-UNet & $89.53$ & $91.63$ & $86.24$ & $\textcolor{db}{\mathbf{90.72}}$ & $4.79$ & $5.05$ & $8.12$ & $4.15$ \\
U-RWKV & $89.04$ & $91.34$ & $86.94$ & $88.83$ & $5.42$ & $6.39$ & $7.66$ & $4.63$ \\
\addlinespace[2pt]
\gr VeloxSeg-C & \gr $\textcolor{dr}{\mathbf{91.44}}$ & \gr $\textcolor{dr}{\mathbf{93.09}}$ & \gr $\textcolor{dr}{\mathbf{89.00}}$ & \gr $\textcolor{dr}{\mathbf{92.24}}$ & \gr $\textcolor{dr}{\mathbf{3.75}}$ & \gr $\textcolor{dr}{\mathbf{3.89}}$ & \gr $\textcolor{dr}{\mathbf{4.41}}$ & \gr $3.35$ \\
\bottomrule
\end{tabular}
\caption{Segmentation performance comparison on the BraTS2021 dataset. VeloxSeg\textnormal{-}C's metrics are highlighted in \colorbox{caption_green}{green}. The best performance is \textcolor{dr}{\textbf{red}} and the second best performance is \textcolor{db}{\textbf{blue}}.}
\label{tab:mri_segmentation_performance}
\end{table}
On BraTS2021 MRI brain tumor dataset, we use an early fusion strategy (VeloxSeg\textnormal{-}C) that does not slow down as the number of modalities increases, as shown in Table~\ref{tab:mri_segmentation_performance}. Since brain tumors are large and centralized, and the slices processed are relatively fixed, almost all models achieved good results. Our model improves the Dice by 1.72\% compared to the state-of-the-art SuperLightNet.
% \section{Device Generalization Evaluation}
% \label{appendix:device_generliaztion_evaluation}
% \begin{table}[h]
% \caption{Results of generalization experiments using BraTS2021 as the training set and MSD2019 as the external test set. VeloxSeg's metrics are highlighted in \colorbox{caption_green}{green}. The best performance is \textcolor{dr}{\textbf{red}} and the second best performance is \textcolor{db}{\textbf{blue}}.}
% \centering
% \small
% \setlength{\tabcolsep}{3pt}
% \renewcommand{\arraystretch}{1.2}
% \label{tab:generalization_ability_evaluation}
% \begin{threeparttable}
% \begin{tabular}{l cccc cccc}
% \toprule
% \multirow{2}{*}{\textbf{Method}} & \multicolumn{4}{c}{\textbf{Dice Similarity Coefficient (\%)}} & \multicolumn{4}{c}{\textbf{Hausdorff Distance 95\% (mm)}} \\
% \cmidrule(lr){2-5}\cmidrule(lr){6-9}
% & \multicolumn{1}{c}{\small\textbf{Avg.} $\uparrow$} & \multicolumn{1}{c}{\small\textbf{ET} $\uparrow$} & \multicolumn{1}{c}{\small\textbf{TC} $\uparrow$} & \multicolumn{1}{c}{\small\textbf{WT} $\uparrow$} & \multicolumn{1}{c}{\small\textbf{Avg.} $\downarrow$} & \multicolumn{1}{c}{\small\textbf{ET} $\downarrow$} & \multicolumn{1}{c}{\small\textbf{TC} $\downarrow$} & \multicolumn{1}{c}{\small\textbf{WT} $\downarrow$} \\
% \midrule
% UNet & $54.63$ & $70.02$ & $47.31$ & $46.56$ & $42.11$ & $54.69$ & $39.22$ & $35.59$ \\
% V-Net & $59.96$ & $71.79$ & $54.67$ & $53.41$ & $43.18$ & $\textcolor{db}{\mathbf{46.83}}$ & $55.56$ & $44.56$ \\
% UNETR & $50.34$ & $65.69$ & $47.43$ & $37.90$ & $\textcolor{dr}{\mathbf{35.81}}$ & $\textcolor{dr}{\mathbf{40.27}}$ & $\textcolor{dr}{\mathbf{30.37}}$ & $38.71$ \\
% Swin UNETR & $59.41$ & $71.27$ & $53.63$ & $53.35$ & $45.32$ & $51.78$ & $58.11$ & $31.67$ \\
% VSmTrans & $58.06$ & $70.69$ & $50.02$ & $53.46$ & $39.79$ & $53.46$ & $\textcolor{db}{\mathbf{35.87}}$ & $34.88$ \\
% UNETR++ & $59.46$ & $\textcolor{db}{\mathbf{72.31}}$ & $54.31$ & $51.75$ & $43.27$ & $50.46$ & $59.96$ & $47.91$ \\
% U-KAN & $60.32$ & $71.81$ & $53.30$ & $55.83$ & $49.58$ & $52.10$ & $55.71$ & $39.74$ \\
% \addlinespace[2pt]
% Nestedformer & $59.65$ & $71.09$ & $53.30$ & $54.54$ & $50.09$ & $63.53$ & $56.47$ & $\textcolor{db}{\mathbf{29.99}}$ \\
% A2FSeg & $\textcolor{dr}{\mathbf{62.84}}$ & $72.22$ & $\textcolor{db}{\mathbf{54.80}}$ & $\textcolor{dr}{\mathbf{61.49}}$ & $\textcolor{db}{\mathbf{37.60}}$ & $51.56$ & $44.40$ & $\textcolor{dr}{\mathbf{23.32}}$ \\
% H-DenseFormer & $59.51$ & $71.81$ & $54.25$ & $52.46$ & $46.77$ & $55.30$ & $52.83$ & $43.02$ \\
% \addlinespace[2pt]
% SegFormer-3D & $60.26$ & $72.09$ & $\textcolor{dr}{\mathbf{54.82}}$ & $53.87$ & $49.49$ & $55.04$ & $57.77$ & $49.48$ \\
% Slim UNETR & $55.83$ & $71.28$ & $49.81$ & $46.39$ & $45.23$ & $57.60$ & $47.84$ & $34.12$ \\
% SuperLightNet & $58.17$ & $71.73$ & $52.42$ & $50.36$ & $50.82$ & $60.47$ & $49.66$ & $43.39$ \\
% HCMA-UNet & $60.09$ & $71.70$ & $53.02$ & $55.56$ & $40.72$ & $52.78$ & $42.90$ & $30.63$ \\
% U-RWKV & $60.25$ & $71.55$ & $53.57$ & $55.64$ & $47.46$ & $56.41$ & $54.23$ & $30.87$ \\
% \addlinespace[2pt]
% \gr VeloxSeg-C & \gr $\textcolor{db}{\mathbf{60.35}}$ & \gr $\textcolor{dr}{\mathbf{72.35}}$ & \gr $54.10$ & \gr $\textcolor{db}{\mathbf{54.61}}$ & \gr $42.92$ & \gr $49.78$ & \gr $47.84$ & \gr $36.24$ \\
% \bottomrule
% \end{tabular}
% \begin{tablenotes}
% \footnotesize
% \item[$$] $P$-value (vs. method marked with $^\ast$) $< 0.05$; $$: $P$-value (vs. method marked with $^\ast$) $< 0.01$.
% \end{tablenotes}
% \end{threeparttable}
% \end{table}
% To evaluate the generalization ability of various 3D segmentation methods, we conduct experiments on two MRI brain tumor datasets, BraTS2021 and MSD2019. Specifically, we complete the training process of all models on the training data of BraTS2021. Then, the parameters of these models are frozen and tested on the entire data of MSD2019. The results are shown in Table~\ref{tab:generalization_ability_evaluation}. Note that the three multimodal methods have high segmentation performance, among which A2FSeg achieved the best average Dice of 62.84\%, demonstrating the advantage of multimodal methods in generalization ability. Since we adopt the early fusion strategy to build the model in BraTS2021, the proposed VeloxSeg\textnormal{-}C achieves competitive results with a mean Dice of 60.35\%.
\section{Hyperparameter Analysis}
\label{appendix:hyperparameter_analysis}
\begin{table}[h]
\centering
% Left table: Pruning results
\begin{minipage}[t]{0.48\textwidth}
\centering
\footnotesize
\setlength{\tabcolsep}{3pt}
\renewcommand{\arraystretch}{1}
\begin{tabular}{ccccc}
\toprule
\textbf{Depth} & \textbf{CT} & \textbf{PET} & \textbf{Enc.} & \textbf{Dec.} \\
\midrule
1 & 3.07 & 3.22 & 3.02 & 3.06 \\
2 & 2.85 & 2.82 & 2.73 & 2.75 \\
3 & 2.37 & 2.38 & 2.26 & 2.29 \\
4 & 1.82 & 1.85 & 1.74 & $-$ \\
\bottomrule
\end{tabular}
\captionof{table}{Ratio of channels to input embeddings after pruning the FFN layers of PWA and JLC. Baseline Dice: 69.94\%; after pruning: 68.49\%.}
\label{tab:prune}
\end{minipage}
\hfill
% Right table: Loss weight ablation
\begin{minipage}[t]{0.48\textwidth}
\centering
\footnotesize
\setlength{\tabcolsep}{3pt}
\renewcommand{\arraystretch}{1}
\begin{tabular}{lccc}
\toprule
\textbf{Ablation} & $\boldsymbol{\lambda_{rc}}$ & $\boldsymbol{\lambda_{style}}$ & \textbf{Dice} $\uparrow$ \\
\midrule
\multirow{3}{*}{$\mathcal{L}_{rc}$} & 1.5 & 1.5 & 58.03 \textcolor{db}{-4.41} \\
& 1.0 & 1.5 & 61.53 \textcolor{db}{-0.92} \\
& \gr0.5 & \gr1.5 & \gr62.44 \\
\midrule
\multirow{5}{*}{$\mathcal{L}_{style}$} & 0.5 & 2.5 & 62.23 \textcolor{db}{-0.28} \\
& \gr0.5 & \gr2.0 & \gr62.51 \\
& 0.5 & 1.5 & 62.44 \textcolor{db}{-0.07} \\
& 0.5 & 1.0 & 61.66 \textcolor{db}{-0.85} \\
& 0.5 & 0.5 & 60.55 \textcolor{db}{-2.06} \\
\bottomrule
\end{tabular}
\captionof{table}{Hyperparameters experiments with loss weight on AutoPET-II.}
\label{tab:multitask_losw_seight_ablation}
\end{minipage}
\end{table}
\subsection{Module Hyperparameter}
Model optimization, detailed in Table~\ref{tab:module_ablation}, focused on balancing segmentation performance and computational efficiency. convolution adjustments, including reducing model width $\langle 32, 61, 128, 256\rangle$ to $\langle 16, 32, 64, 128\rangle$, improved CPU throughput, from 10.83 to 20.23, and Dice, from 48.96\% to 50.10\%. Replacing large kernel convolution $\langle 7\rangle$ with parallel small kernels $\langle 1,3,5\rangle$ yielded a 3.55\% Dice increase, from 50.10\% to 53.65\%, while simultaneously reducing MParams from 0.73 to 0.66, and GFLOPs from 2.41 to 2.30. Optimal group channel setting $\langle 4,8,8,16\rangle$ achieved a 55.14\% Dice. Attention depth reduction $\langle 2,2,2,2\rangle$ to $\langle 1,1,1,1\rangle$ surprisingly enhanced Dice, from 59.56\% to 61.03\%. As suggested in Table~\ref{tab:prune}, reducing the FFN dilation rate of Transformer/Convolution to $\langle 3,3,2,2\rangle$ can slightly improve Dice performance while reducing computational cost.
\subsection{Loss Weight}
Considering the differences in the contributions of various tasks to segmentation, we need to adjust the relative weights between different tasks to explore the optimal parameter update process. To this end, we adopt the strategy of controlling variables and adjust the loss weights of $\mathcal{L}_{rc}$ and $\mathcal{L}_{style}$ in turn. The specific results are shown in Table~\ref{tab:multitask_losw_seight_ablation}. The final parameters of each experiment are highlighted in \colorbox{caption_green}{green}, and red and green are used to indicate the improvement and deterioration in the process. Finally, the optimal weight parameters are selected as $\lambda_{rc}=0.5$, $\lambda_{style}=2.0$.
\section{Computation of Mean Attention Distance}
\label{appendix:computation_mean_attention_distance}
To analyze the locality of attention heads, we compute the Mean Attention Distance (MAD), a metric that measures the average physical distance between a query voxel and the key voxels it attends to, weighted by the attention scores. We extend the Mean Attention Distance metric to 3D volumes to analyze attention patterns in volumetric data, and its computation is detailed below.
Let the input 3D volume be partitioned into a grid of $H \times W \times D$ voxels, resulting in a total of $L = H \cdot W \cdot D$ voxels. The attention weight matrix for a given head is denoted by $\mathbf{W} \in \mathbb{R}^{L \times L}$, where $\mathbf{W} _{ij}$ is the attention weight from voxel $i$ to voxel $j$. Let $\mathbf{s} \in \mathbb{R}^+$ be a scalar representing the physical edge length of a single cubic voxel.
First, we map each voxel's flattened 1D index $i$ (where $i \in \{0, \dots, L-1\}$) to its physical 3D coordinates $(x_i, y_i, z_i)$. Assuming a standard row-major flattening order where the x-axis (width) is the fastest-changing dimension and the z-axis (depth) is the slowest:
\begin{equation}
\begin{cases}
z_i=\lfloor i\,\,/ (H\cdot W) \rfloor\\
y_i=\lfloor (i\,\,\mathrm{mod}\;\left( H\cdot W \right) ) / W \\
x_i=i\,\,\mathrm{mod}\;W\\
\end{cases}.
\end{equation}
Next, we compute the pairwise physical distance matrix $\mathbf{D} \in \mathbb{R}^{L \times L}$. Each element $\mathbf{D}_{ij}$ represents the scaled Euclidean distance between the centers of voxel $i$ and voxel $j$:
\begin{equation}
\mathbf{D}_{ij} = \mathbf{s} \cdot \sqrt{(x_i - x_j)^2 + (y_i - y_j)^2 + (z_i - z_j)^2}
\label{eq:distance_matrix_3d_hwd}
\end{equation}
Finally, the Mean Attention Distance (MAD) is formulated as the expectation of the physical distance over the attention distribution. As shown in Equation~\ref{eq:mad_3d_hwd}, it is computed by summing all distances weighted by their corresponding attention scores, and then averaging over all query voxels:
\begin{equation}
\text{MAD} = \frac{1}{L} \sum_{i=0}^{L-1} \sum_{j=0}^{L-1} \mathbf{W}_{ij} \cdot \mathbf{D}_{ij}
\label{eq:mad_3d_hwd}
\end{equation}
A smaller MAD value indicates that the attention head primarily focuses on local information, whereas a larger MAD value signifies a more global attention pattern across the volume. The window size of each stage of PWA is shown in Appendix~\ref{appendix:details_PWA}.
\section{PWA MultiModal Evaluation}
\label{appendix:pwa_multimodal_evaluation}
\begin{table}[h]
\centering
\small
\setlength{\tabcolsep}{3pt}
\renewcommand{\arraystretch}{1}
\begin{tabular}{lcccc}
\toprule
\textbf{Modality} & \textbf{MParams} $\downarrow$ & \textbf{GFLOPs} $\downarrow$ & \textbf{Thr. (GPU)} $\uparrow$ & \textbf{Dice} $\uparrow$ \\
\midrule
CT & 1.39 \textcolor{dr}{-0.27} & 1.61 \textcolor{dr}{-0.18} & 1038.94 \textcolor{dr}{+439.88} & 21.43 \textcolor{db}{-41.01} \\
PET & 1.39 \textcolor{dr}{-0.27} & 1.61 \textcolor{dr}{-0.18} & 1038.94 \textcolor{dr}{+439.88} & 49.28 \textcolor{db}{-13.16} \\
PET + CT & 1.39 \textcolor{dr}{-0.27} & 1.70 \textcolor{dr}{-0.09} & 963.22 \textcolor{dr}{+364.16} & 56.69 \textcolor{db}{-5.75} \\
\gr$\langle \text{CT, PET}\rangle$ & \gr1.66 & \gr1.79 & \gr599.06 & \gr62.51 \\
\bottomrule
\end{tabular}
\caption{Modality ablation experiments performed on AutoPET-II. ``PET+CT'' indicates an early fusion strategy, and ``$\langle \text{CT, PET}\rangle$'' indicates consideration of modality interaction.}
\label{tab:modality_ablation}
\end{table}
To verify the effectiveness of PWA in heterogeneous modal modeling, we test various inputs, as shown in Table~\ref{tab:modality_ablation}. Using only PET or CT reduces model size and complexity but sacrifices segmentation performance. An early fusion strategy achieves a Dice of 56.69\%, outperforming the pure convolution framework's 55.84\%. Crucially, introducing modal interaction in PWA improves Dice by 5.75\%, significantly improving performance robustness without significantly increasing computational or time costs.
\section{JL-Setting Generalization Evaluation}
\label{appendix:jl_comparison_hecktor_brast}
\begin{table}[h]
\centering
\caption{Performance comparison of different group size configurations across datasets. The JL-guided configuration $\langle 4,8,8,16\rangle$ is used as the reference baseline.}
\label{tab:jl_generalization}
\small
\setlength{\tabcolsep}{3pt}
\renewcommand{\arraystretch}{1.2}
\begin{tabular}{l l c c c}
\toprule
\textbf{Dataset} & \textbf{Configuration} & \textbf{MParams} $\downarrow$ & \textbf{GFLOPs} $\downarrow$ & \textbf{Dice} $\uparrow$ \\
\midrule
\multirow{3}{*}{Hecktor2022} & $\langle 1,1,1,1\rangle$ & 0.618 \textcolor{dr}{-0.091} & 2.637 \textcolor{dr}{-0.075} & 37.95 \textcolor{db}{-6.25} \\
& \gr$\langle 4,8,8,16\rangle$ & \gr0.709 & \gr2.712 & \gr44.20 \\
& $\langle 16,32,64,128\rangle$ & 1.342 \textcolor{db}{+0.633} & 3.029 \textcolor{db}{+0.317} & 43.21\textcolor{db}{-0.99} \\
\midrule
\multirow{3}{*}{BraTS2021} & $\langle 1,1,1,1\rangle$ & 0.629 \textcolor{dr}{-0.091} & 2.377 \textcolor{dr}{-0.063} & 85.82 \textcolor{db}{-1.00} \\
& \gr$\langle 4,8,8,16\rangle$ & \gr0.720 & \gr2.440 & \gr86.82 \\
& $\langle 16,32,64,128\rangle$ & 1.353 \textcolor{db}{+0.633} & 2.708 \textcolor{db}{+0.268} & 87.98 \textcolor{dr}{+1.16} \\
\bottomrule
\end{tabular}
\end{table}
Results demonstrate the effectiveness of JL-guided group size configurations $\langle 4,8,8,16\rangle$ on various datasets. While the smallest configuration $\langle 1,1,1,1\rangle$ achieves the lowest computational cost, reducing segmentation performance by 0.091 MParams and 0.063 to 0.075 GFLOPs, it significantly degrades segmentation performance, particularly on the Hecktor2022 dataset, which features heterogeneous modality data and cross-organ distribution of targets, where the Dice drops by 6.25\%. Larger configurations $\langle 16,32,64,128\rangle$ only slightly improve the Dice (by 1.16\% on BraTS2021) but significantly increase computational complexity by 0.633 MParams and 0.268 to 0.317 GFLOPs. This experiment further demonstrates that JL-guided configurations strike an optimal balance, maintaining competitive performance while ensuring computational efficiency suitable for clinical deployment.
\end{document}
%%%%% NEW MATH DEFINITIONS %%%%%
\usepackage{amsmath,amsfonts,bm}
% Mark sections of captions for referring to divisions of figures
\newcommand{\figleft}{{\em (Left)}}
\newcommand{\figcenter}{{\em (Center)}}
\newcommand{\figright}{{\em (Right)}}
\newcommand{\figtop}{{\em (Top)}}
\newcommand{\figbottom}{{\em (Bottom)}}
\newcommand{\captiona}{{\em (a)}}
\newcommand{\captionb}{{\em (b)}}
\newcommand{\captionc}{{\em (c)}}
\newcommand{\captiond}{{\em (d)}}
% Highlight a newly defined term
\newcommand{\newterm}[1]{{\bf #1}}
% Figure reference, lower-case.
\def\figref#1{figure~\ref{#1}}
% Figure reference, capital. For start of sentence
\def\Figref#1{Figure~\ref{#1}}
\def\twofigref#1#2{figures \ref{#1} and \ref{#2}}
\def\quadfigref#1#2#3#4{figures \ref{#1}, \ref{#2}, \ref{#3} and \ref{#4}}
% Section reference, lower-case.
\def\secref#1{section~\ref{#1}}
% Section reference, capital.
\def\Secref#1{Section~\ref{#1}}
% Reference to two sections.
\def\twosecrefs#1#2{sections \ref{#1} and \ref{#2}}
% Reference to three sections.
\def\secrefs#1#2#3{sections \ref{#1}, \ref{#2} and \ref{#3}}
% Reference to an equation, lower-case.
\def\eqref#1{equation~\ref{#1}}
% Reference to an equation, upper case
\def\Eqref#1{Equation~\ref{#1}}
% A raw reference to an equation---avoid using if possible
\def\plaineqref#1{\ref{#1}}
% Reference to a chapter, lower-case.
\def\chapref#1{chapter~\ref{#1}}
% Reference to an equation, upper case.
\def\Chapref#1{Chapter~\ref{#1}}
% Reference to a range of chapters
\def\rangechapref#1#2{chapters\ref{#1}--\ref{#2}}
% Reference to an algorithm, lower-case.
\def\algref#1{algorithm~\ref{#1}}
% Reference to an algorithm, upper case.
\def\Algref#1{Algorithm~\ref{#1}}
\def\twoalgref#1#2{algorithms \ref{#1} and \ref{#2}}
\def\Twoalgref#1#2{Algorithms \ref{#1} and \ref{#2}}
% Reference to a part, lower case
\def\partref#1{part~\ref{#1}}
% Reference to a part, upper case
\def\Partref#1{Part~\ref{#1}}
\def\twopartref#1#2{parts \ref{#1} and \ref{#2}}
\def\ceil#1{\lceil #1 \rceil}
\def\floor#1{\lfloor #1 \rfloor}
\def\1{\bm{1}}
\newcommand{\train}{\mathcal{D}}
\newcommand{\valid}{\mathcal{D_{\mathrm{valid}}}}
\newcommand{\test}{\mathcal{D_{\mathrm{test}}}}
\def\eps{{\epsilon}}
% Random variables
\def\reta{{\textnormal{$\eta$}}}
\def\ra{{\textnormal{a}}}
\def\rb{{\textnormal{b}}}
\def\rc{{\textnormal{c}}}
\def\rd{{\textnormal{d}}}
\def\re{{\textnormal{e}}}
\def\rf{{\textnormal{f}}}
\def\rg{{\textnormal{g}}}
\def\rh{{\textnormal{h}}}
\def\ri{{\textnormal{i}}}
\def\rj{{\textnormal{j}}}
\def\rk{{\textnormal{k}}}
\def\rl{{\textnormal{l}}}
% rm is already a command, just don't name any random variables m
\def\rn{{\textnormal{n}}}
\def\ro{{\textnormal{o}}}
\def\rp{{\textnormal{p}}}
\def\rq{{\textnormal{q}}}
\def\rr{{\textnormal{r}}}
\def\rs{{\textnormal{s}}}
\def\rt{{\textnormal{t}}}
\def\ru{{\textnormal{u}}}
\def\rv{{\textnormal{v}}}
\def\rw{{\textnormal{w}}}
\def\rx{{\textnormal{x}}}
\def\ry{{\textnormal{y}}}
\def\rz{{\textnormal{z}}}
% Random vectors
\def\rvepsilon{{\mathbf{\epsilon}}}
\def\rvtheta{{\mathbf{\theta}}}
\def\rva{{\mathbf{a}}}
\def\rvb{{\mathbf{b}}}
\def\rvc{{\mathbf{c}}}
\def\rvd{{\mathbf{d}}}
\def\rve{{\mathbf{e}}}
\def\rvf{{\mathbf{f}}}
\def\rvg{{\mathbf{g}}}
\def\rvh{{\mathbf{h}}}
\def\rvu{{\mathbf{i}}}
\def\rvj{{\mathbf{j}}}
\def\rvk{{\mathbf{k}}}
\def\rvl{{\mathbf{l}}}
\def\rvm{{\mathbf{m}}}
\def\rvn{{\mathbf{n}}}
\def\rvo{{\mathbf{o}}}
\def\rvp{{\mathbf{p}}}
\def\rvq{{\mathbf{q}}}
\def\rvr{{\mathbf{r}}}
\def\rvs{{\mathbf{s}}}
\def\rvt{{\mathbf{t}}}
\def\rvu{{\mathbf{u}}}
\def\rvv{{\mathbf{v}}}
\def\rvw{{\mathbf{w}}}
\def\rvx{{\mathbf{x}}}
\def\rvy{{\mathbf{y}}}
\def\rvz{{\mathbf{z}}}
% Elements of random vectors
\def\erva{{\textnormal{a}}}
\def\ervb{{\textnormal{b}}}
\def\ervc{{\textnormal{c}}}
\def\ervd{{\textnormal{d}}}
\def\erve{{\textnormal{e}}}
\def\ervf{{\textnormal{f}}}
\def\ervg{{\textnormal{g}}}
\def\ervh{{\textnormal{h}}}
\def\ervi{{\textnormal{i}}}
\def\ervj{{\textnormal{j}}}
\def\ervk{{\textnormal{k}}}
\def\ervl{{\textnormal{l}}}
\def\ervm{{\textnormal{m}}}
\def\ervn{{\textnormal{n}}}
\def\ervo{{\textnormal{o}}}
\def\ervp{{\textnormal{p}}}
\def\ervq{{\textnormal{q}}}
\def\ervr{{\textnormal{r}}}
\def\ervs{{\textnormal{s}}}
\def\ervt{{\textnormal{t}}}
\def\ervu{{\textnormal{u}}}
\def\ervv{{\textnormal{v}}}
\def\ervw{{\textnormal{w}}}
\def\ervx{{\textnormal{x}}}
\def\ervy{{\textnormal{y}}}
\def\ervz{{\textnormal{z}}}
% Random matrices
\def\rmA{{\mathbf{A}}}
\def\rmB{{\mathbf{B}}}
\def\rmC{{\mathbf{C}}}
\def\rmD{{\mathbf{D}}}
\def\rmE{{\mathbf{E}}}
\def\rmF{{\mathbf{F}}}
\def\rmG{{\mathbf{G}}}
\def\rmH{{\mathbf{H}}}
\def\rmI{{\mathbf{I}}}
\def\rmJ{{\mathbf{J}}}
\def\rmK{{\mathbf{K}}}
\def\rmL{{\mathbf{L}}}
\def\rmM{{\mathbf{M}}}
\def\rmN{{\mathbf{N}}}
\def\rmO{{\mathbf{O}}}
\def\rmP{{\mathbf{P}}}
\def\rmQ{{\mathbf{Q}}}
\def\rmR{{\mathbf{R}}}
\def\rmS{{\mathbf{S}}}
\def\rmT{{\mathbf{T}}}
\def\rmU{{\mathbf{U}}}
\def\rmV{{\mathbf{V}}}
\def\rmW{{\mathbf{W}}}
\def\rmX{{\mathbf{X}}}
\def\rmY{{\mathbf{Y}}}
\def\rmZ{{\mathbf{Z}}}
% Elements of random matrices
\def\ermA{{\textnormal{A}}}
\def\ermB{{\textnormal{B}}}
\def\ermC{{\textnormal{C}}}
\def\ermD{{\textnormal{D}}}
\def\ermE{{\textnormal{E}}}
\def\ermF{{\textnormal{F}}}
\def\ermG{{\textnormal{G}}}
\def\ermH{{\textnormal{H}}}
\def\ermI{{\textnormal{I}}}
\def\ermJ{{\textnormal{J}}}
\def\ermK{{\textnormal{K}}}
\def\ermL{{\textnormal{L}}}
\def\ermM{{\textnormal{M}}}
\def\ermN{{\textnormal{N}}}
\def\ermO{{\textnormal{O}}}
\def\ermP{{\textnormal{P}}}
\def\ermQ{{\textnormal{Q}}}
\def\ermR{{\textnormal{R}}}
\def\ermS{{\textnormal{S}}}
\def\ermT{{\textnormal{T}}}
\def\ermU{{\textnormal{U}}}
\def\ermV{{\textnormal{V}}}
\def\ermW{{\textnormal{W}}}
\def\ermX{{\textnormal{X}}}
\def\ermY{{\textnormal{Y}}}
\def\ermZ{{\textnormal{Z}}}
% Vectors
\def\vzero{{\bm{0}}}
\def\vone{{\bm{1}}}
\def\vmu{{\bm{\mu}}}
\def\vtheta{{\bm{\theta}}}
\def\va{{\bm{a}}}
\def\vb{{\bm{b}}}
\def\vc{{\bm{c}}}
\def\vd{{\bm{d}}}
\def\ve{{\bm{e}}}
\def\vf{{\bm{f}}}
\def\vg{{\bm{g}}}
\def\vh{{\bm{h}}}
\def\vi{{\bm{i}}}
\def\vj{{\bm{j}}}
\def\vk{{\bm{k}}}
\def\vl{{\bm{l}}}
\def\vm{{\bm{m}}}
\def\vn{{\bm{n}}}
\def\vo{{\bm{o}}}
\def\vp{{\bm{p}}}
\def\vq{{\bm{q}}}
\def\vr{{\bm{r}}}
\def\vs{{\bm{s}}}
\def\vt{{\bm{t}}}
\def\vu{{\bm{u}}}
\def\vv{{\bm{v}}}
\def\vw{{\bm{w}}}
\def\vx{{\bm{x}}}
\def\vy{{\bm{y}}}
\def\vz{{\bm{z}}}
% Elements of vectors
\def\evalpha{{\alpha}}
\def\evbeta{{\beta}}
\def\evepsilon{{\epsilon}}
\def\evlambda{{\lambda}}
\def\evomega{{\omega}}
\def\evmu{{\mu}}
\def\evpsi{{\psi}}
\def\evsigma{{\sigma}}
\def\evtheta{{\theta}}
\def\eva{{a}}
\def\evb{{b}}
\def\evc{{c}}
\def\evd{{d}}
\def\eve{{e}}
\def\evf{{f}}
\def\evg{{g}}
\def\evh{{h}}
\def\evi{{i}}
\def\evj{{j}}
\def\evk{{k}}
\def\evl{{l}}
\def\evm{{m}}
\def\evn{{n}}
\def\evo{{o}}
\def\evp{{p}}
\def\evq{{q}}
\def\evr{{r}}
\def\evs{{s}}
\def\evt{{t}}
\def\evu{{u}}
\def\evv{{v}}
\def\evw{{w}}
\def\evx{{x}}
\def\evy{{y}}
\def\evz{{z}}
% Matrix
\def\mA{{\bm{A}}}
\def\mB{{\bm{B}}}
\def\mC{{\bm{C}}}
\def\mD{{\bm{D}}}
\def\mE{{\bm{E}}}
\def\mF{{\bm{F}}}
\def\mG{{\bm{G}}}
\def\mH{{\bm{H}}}
\def\mI{{\bm{I}}}
\def\mJ{{\bm{J}}}
\def\mK{{\bm{K}}}
\def\mL{{\bm{L}}}
\def\mM{{\bm{M}}}
\def\mN{{\bm{N}}}
\def\mO{{\bm{O}}}
\def\mP{{\bm{P}}}
\def\mQ{{\bm{Q}}}
\def\mR{{\bm{R}}}
\def\mS{{\bm{S}}}
\def\mT{{\bm{T}}}
\def\mU{{\bm{U}}}
\def\mV{{\bm{V}}}
\def\mW{{\bm{W}}}
\def\mX{{\bm{X}}}
\def\mY{{\bm{Y}}}
\def\mZ{{\bm{Z}}}
\def\mBeta{{\bm{\beta}}}
\def\mPhi{{\bm{\Phi}}}
\def\mLambda{{\bm{\Lambda}}}
\def\mSigma{{\bm{\Sigma}}}
% Tensor
\DeclareMathAlphabet{\mathsfit}{\encodingdefault}{\sfdefault}{m}{sl}
\SetMathAlphabet{\mathsfit}{bold}{\encodingdefault}{\sfdefault}{bx}{n}
\newcommand{\tens}[1]{\bm{\mathsfit{#1}}}
\def\tA{{\tens{A}}}
\def\tB{{\tens{B}}}
\def\tC{{\tens{C}}}
\def\tD{{\tens{D}}}
\def\tE{{\tens{E}}}
\def\tF{{\tens{F}}}
\def\tG{{\tens{G}}}
\def\tH{{\tens{H}}}
\def\tI{{\tens{I}}}
\def\tJ{{\tens{J}}}
\def\tK{{\tens{K}}}
\def\tL{{\tens{L}}}
\def\tM{{\tens{M}}}
\def\tN{{\tens{N}}}
\def\tO{{\tens{O}}}
\def\tP{{\tens{P}}}
\def\tQ{{\tens{Q}}}
\def\tR{{\tens{R}}}
\def\tS{{\tens{S}}}
\def\tT{{\tens{T}}}
\def\tU{{\tens{U}}}
\def\tV{{\tens{V}}}
\def\tW{{\tens{W}}}
\def\tX{{\tens{X}}}
\def\tY{{\tens{Y}}}
\def\tZ{{\tens{Z}}}
% Graph
\def\gA{{\mathcal{A}}}
\def\gB{{\mathcal{B}}}
\def\gC{{\mathcal{C}}}
\def\gD{{\mathcal{D}}}
\def\gE{{\mathcal{E}}}
\def\gF{{\mathcal{F}}}
\def\gG{{\mathcal{G}}}
\def\gH{{\mathcal{H}}}
\def\gI{{\mathcal{I}}}
\def\gJ{{\mathcal{J}}}
\def\gK{{\mathcal{K}}}
\def\gL{{\mathcal{L}}}
\def\gM{{\mathcal{M}}}
\def\gN{{\mathcal{N}}}
\def\gO{{\mathcal{O}}}
\def\gP{{\mathcal{P}}}
\def\gQ{{\mathcal{Q}}}
\def\gR{{\mathcal{R}}}
\def\gS{{\mathcal{S}}}
\def\gT{{\mathcal{T}}}
\def\gU{{\mathcal{U}}}
\def\gV{{\mathcal{V}}}
\def\gW{{\mathcal{W}}}
\def\gX{{\mathcal{X}}}
\def\gY{{\mathcal{Y}}}
\def\gZ{{\mathcal{Z}}}
% Sets
\def\sA{{\mathbb{A}}}
\def\sB{{\mathbb{B}}}
\def\sC{{\mathbb{C}}}
\def\sD{{\mathbb{D}}}
% Don't use a set called E, because this would be the same as our symbol
% for expectation.
\def\sF{{\mathbb{F}}}
\def\sG{{\mathbb{G}}}
\def\sH{{\mathbb{H}}}
\def\sI{{\mathbb{I}}}
\def\sJ{{\mathbb{J}}}
\def\sK{{\mathbb{K}}}
\def\sL{{\mathbb{L}}}
\def\sM{{\mathbb{M}}}
\def\sN{{\mathbb{N}}}
\def\sO{{\mathbb{O}}}
\def\sP{{\mathbb{P}}}
\def\sQ{{\mathbb{Q}}}
\def\sR{{\mathbb{R}}}
\def\sS{{\mathbb{S}}}
\def\sT{{\mathbb{T}}}
\def\sU{{\mathbb{U}}}
\def\sV{{\mathbb{V}}}
\def\sW{{\mathbb{W}}}
\def\sX{{\mathbb{X}}}
\def\sY{{\mathbb{Y}}}
\def\sZ{{\mathbb{Z}}}
% Entries of a matrix
\def\emLambda{{\Lambda}}
\def\emA{{A}}
\def\emB{{B}}
\def\emC{{C}}
\def\emD{{D}}
\def\emE{{E}}
\def\emF{{F}}
\def\emG{{G}}
\def\emH{{H}}
\def\emI{{I}}
\def\emJ{{J}}
\def\emK{{K}}
\def\emL{{L}}
\def\emM{{M}}
\def\emN{{N}}
\def\emO{{O}}
\def\emP{{P}}
\def\emQ{{Q}}
\def\emR{{R}}
\def\emS{{S}}
\def\emT{{T}}
\def\emU{{U}}
\def\emV{{V}}
\def\emW{{W}}
\def\emX{{X}}
\def\emY{{Y}}
\def\emZ{{Z}}
\def\emSigma{{\Sigma}}
% entries of a tensor
% Same font as tensor, without \bm wrapper
\newcommand{\etens}[1]{\mathsfit{#1}}
\def\etLambda{{\etens{\Lambda}}}
\def\etA{{\etens{A}}}
\def\etB{{\etens{B}}}
\def\etC{{\etens{C}}}
\def\etD{{\etens{D}}}
\def\etE{{\etens{E}}}
\def\etF{{\etens{F}}}
\def\etG{{\etens{G}}}
\def\etH{{\etens{H}}}
\def\etI{{\etens{I}}}
\def\etJ{{\etens{J}}}
\def\etK{{\etens{K}}}
\def\etL{{\etens{L}}}
\def\etM{{\etens{M}}}
\def\etN{{\etens{N}}}
\def\etO{{\etens{O}}}
\def\etP{{\etens{P}}}
\def\etQ{{\etens{Q}}}
\def\etR{{\etens{R}}}
\def\etS{{\etens{S}}}
\def\etT{{\etens{T}}}
\def\etU{{\etens{U}}}
\def\etV{{\etens{V}}}
\def\etW{{\etens{W}}}
\def\etX{{\etens{X}}}
\def\etY{{\etens{Y}}}
\def\etZ{{\etens{Z}}}
% The true underlying data generating distribution
\newcommand{\pdata}{p_{\rm{data}}}
% The empirical distribution defined by the training set
\newcommand{\ptrain}{\hat{p}_{\rm{data}}}
\newcommand{\Ptrain}{\hat{P}_{\rm{data}}}
% The model distribution
\newcommand{\pmodel}{p_{\rm{model}}}
\newcommand{\Pmodel}{P_{\rm{model}}}
\newcommand{\ptildemodel}{\tilde{p}_{\rm{model}}}
% Stochastic autoencoder distributions
\newcommand{\pencode}{p_{\rm{encoder}}}
\newcommand{\pdecode}{p_{\rm{decoder}}}
\newcommand{\precons}{p_{\rm{reconstruct}}}
\newcommand{\laplace}{\mathrm{Laplace}} % Laplace distribution
\newcommand{\E}{\mathbb{E}}
\newcommand{\Ls}{\mathcal{L}}
\newcommand{\R}{\mathbb{R}}
\newcommand{\emp}{\tilde{p}}
\newcommand{\lr}{\alpha}
\newcommand{\reg}{\lambda}
\newcommand{\rect}{\mathrm{rectifier}}
\newcommand{\softmax}{\mathrm{softmax}}
\newcommand{\sigmoid}{\sigma}
\newcommand{\softplus}{\zeta}
\newcommand{\KL}{D_{\mathrm{KL}}}
\newcommand{\Var}{\mathrm{Var}}
\newcommand{\standarderror}{\mathrm{SE}}
\newcommand{\Cov}{\mathrm{Cov}}
% Wolfram Mathworld says $L^2$ is for function spaces and $\ell^2$ is for vectors
% But then they seem to use $L^2$ for vectors throughout the site, and so does
% wikipedia.
\newcommand{\normlzero}{L^0}
\newcommand{\normlone}{L^1}
\newcommand{\normltwo}{L^2}
\newcommand{\normlp}{L^p}
\newcommand{\normmax}{L^\infty}
\newcommand{\parents}{Pa} % See usage in notation.tex. Chosen to match Daphne's book.
\DeclareMathOperator*{\argmax}{arg\,max}
\DeclareMathOperator*{\argmin}{arg\,min}
\DeclareMathOperator{\sign}{sign}
\DeclareMathOperator{\Tr}{Tr}
\let\ab\allowbreak
\begin{thebibliography}{79}
\providecommand{\natexlab}[1]{#1}
\providecommand{\url}[1]{\texttt{#1}}
\expandafter\ifx\csname urlstyle\endcsname\relax
\providecommand{\doi}[1]{doi: #1}\else
\providecommand{\doi}{doi: \begingroup \urlstyle{rm}\Url}\fi
\bibitem[lec(1989)]{lecun_cnn}
Backpropagation applied to handwritten zip code recognition.
\newblock \emph{Neural computation}, 1\penalty0 (4):\penalty0 541--551, 1989.
\bibitem[Akiva et~al.(2022)Akiva, Purri, and Leotta]{ssl_texture}
Peri Akiva, Matthew Purri, and Matthew Leotta.
\newblock Self-supervised material and texture representation learning for remote sensing tasks.
\newblock In \emph{Proceedings of the IEEE/CVF conference on computer vision and pattern recognition}, pp.\ 8203--8215, 2022.
\bibitem[Baid et~al.(2021)]{BraTS2021}
Ujjwal Baid et~al.
\newblock The rsna-asnr-miccai brats 2021 benchmark on brain tumor segmentation and radiogenomic classification.
\newblock \emph{arXiv preprint arXiv:2107.02314}, 2021.
\bibitem[Bakas et~al.(2017)]{mri2}
Spyridon Bakas et~al.
\newblock Advancing the cancer genome atlas glioma mri collections with expert segmentation labels and radiomic features.
\newblock \emph{Scientific data}, 4\penalty0 (1):\penalty0 1--13, 2017.
\bibitem[Bussink et~al.(2011)Bussink, Kaanders, Van Der~Graaf, and Oyen]{pet/ct3}
Johan Bussink, Johannes~HAM Kaanders, Winette~TA Van Der~Graaf, and Wim~JG Oyen.
\newblock Pet--ct for radiotherapy treatment planning and response monitoring in solid tumors.
\newblock \emph{Nature Reviews Clinical Oncology}, 8\penalty0 (4):\penalty0 233--242, 2011.
\bibitem[Chen et~al.(2020)Chen, Li, Yu, El~Kholy, Ahmed, Gan, Cheng, and Liu]{attn_text_image2}
Yen-Chun Chen, Linjie Li, Licheng Yu, Ahmed El~Kholy, Faisal Ahmed, Zhe Gan, Yu~Cheng, and Jingjing Liu.
\newblock Uniter: Universal image-text representation learning.
\newblock In \emph{European conference on computer vision}, pp.\ 104--120. Springer, 2020.
\bibitem[Chollet(2017)]{xception}
Fran{\c{c}}ois Chollet.
\newblock Xception: Deep learning with depthwise separable convolutions.
\newblock In \emph{Proceedings of the IEEE conference on computer vision and pattern recognition}, pp.\ 1251--1258, 2017.
\bibitem[{\c{C}}i{\c{c}}ek et~al.(2016){\c{C}}i{\c{c}}ek, Abdulkadir, Lienkamp, Brox, and Ronneberger]{unet3d}
{\"O}zg{\"u}n {\c{C}}i{\c{c}}ek, Ahmed Abdulkadir, Soeren~S Lienkamp, Thomas Brox, and Olaf Ronneberger.
\newblock 3d u-net: learning dense volumetric segmentation from sparse annotation.
\newblock In \emph{Medical Image Computing and Computer-Assisted Intervention--MICCAI 2016: 19th International Conference, Athens, Greece, October 17-21, 2016, Proceedings, Part II 19}, pp.\ 424--432. Springer, 2016.
\bibitem[Dosovitskiy et~al.(2020)Dosovitskiy, Beyer, Kolesnikov, Weissenborn, Zhai, Unterthiner, Dehghani, Minderer, Heigold, Gelly, et~al.]{attn_text_image3}
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et~al.
\newblock An image is worth 16x16 words: Transformers for image recognition at scale.
\newblock \emph{arXiv preprint arXiv:2010.11929}, 2020.
\bibitem[Du et~al.(2025)Du, Dong, Xu, and Liao]{du2025tdformer}
Hao Du, Qihua Dong, Yan Xu, and Jing Liao.
\newblock Tdformer: Top-down token generation for 3d medical image segmentation.
\newblock \emph{IEEE Journal of Biomedical and Health Informatics}, 2025.
\bibitem[Fang et~al.(2023)Fang, Ma, Song, Mi, and Wang]{torch_pruning}
Gongfan Fang, Xinyin Ma, Mingli Song, Michael~Bi Mi, and Xinchao Wang.
\newblock Depgraph: Towards any structural pruning.
\newblock In \emph{Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition}, pp.\ 16091--16101, 2023.
\bibitem[Filters’Importance(2016)]{l2_pruning}
Determine Filters’Importance.
\newblock Pruning filters for efficient convnets.
\newblock \emph{arXiv preprint arXiv:1608.08710}, 2016.
\bibitem[Gatidis~S(2022)]{AutoPET-II}
Kuestner~T. Gatidis~S.
\newblock A whole-body fdg-pet/ct dataset with manually annotated tumor lesions (fdg-pet-ct-lesions).
\newblock \emph{The Cancer Imaging Archive}, 226, 2022.
\bibitem[Gatys et~al.(2015)Gatys, Ecker, and Bethge]{gram2}
Leon Gatys, Alexander~S Ecker, and Matthias Bethge.
\newblock Texture synthesis using convolutional neural networks.
\newblock \emph{Advances in neural information processing systems}, 28, 2015.
\bibitem[Gatys et~al.(2016)Gatys, Ecker, and Bethge]{gram1}
Leon~A Gatys, Alexander~S Ecker, and Matthias Bethge.
\newblock Image style transfer using convolutional neural networks.
\newblock In \emph{Proceedings of the IEEE conference on computer vision and pattern recognition}, pp.\ 2414--2423, 2016.
\bibitem[Gu \& Dao(2023)Gu and Dao]{mamba}
Albert Gu and Tri Dao.
\newblock Mamba: Linear-time sequence modeling with selective state spaces.
\newblock \emph{arXiv preprint arXiv:2312.00752}, 2023.
\bibitem[Gupta et~al.(2017)Gupta, Johnson, Alahi, and Fei-Fei]{gram_math2}
Agrim Gupta, Justin Johnson, Alexandre Alahi, and Li~Fei-Fei.
\newblock Characterizing and improving stability in neural style transfer.
\newblock In \emph{Proceedings of the IEEE International Conference on Computer Vision}, pp.\ 4067--4076, 2017.
\bibitem[Hatamizadeh et~al.(2021)Hatamizadeh, Nath, Tang, Yang, Roth, and Xu]{swinunetr}
Ali Hatamizadeh, Vishwesh Nath, Yucheng Tang, Dong Yang, Holger~R Roth, and Daguang Xu.
\newblock Swin unetr: Swin transformers for semantic segmentation of brain tumors in mri images.
\newblock In \emph{International MICCAI Brainlesion Workshop}, pp.\ 272--284. Springer, 2021.
\bibitem[Hatamizadeh et~al.(2022)]{unetr}
Ali Hatamizadeh et~al.
\newblock Unetr: Transformers for 3d medical image segmentation.
\newblock In \emph{Proceedings of the IEEE/CVF winter conference on applications of computer vision}, pp.\ 574--584, 2022.
\bibitem[He et~al.(2023)He, Wang, Li, Du, Xia, and Fu]{h2former}
Along He, Kai Wang, Tao Li, Chengkun Du, Shuang Xia, and Huazhu Fu.
\newblock H2former: An efficient hierarchical hybrid transformer for medical image segmentation.
\newblock \emph{IEEE Transactions on Medical Imaging}, 42\penalty0 (9):\penalty0 2763--2775, 2023.
\bibitem[He et~al.(2016)He, Zhang, Ren, and Sun]{resnet}
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun.
\newblock Deep residual learning for image recognition.
\newblock In \emph{Proceedings of the IEEE conference on computer vision and pattern recognition}, pp.\ 770--778, 2016.
\bibitem[He et~al.(2025)He, Guo, Tang, Myronenko, Nath, Xu, Yang, Zhao, Simon, Belue, et~al.]{he2025vista3d}
Yufan He, Pengfei Guo, Yucheng Tang, Andriy Myronenko, Vishwesh Nath, Ziyue Xu, Dong Yang, Can Zhao, Benjamin Simon, Mason Belue, et~al.
\newblock Vista3d: A unified segmentation foundation model for 3d medical imaging.
\newblock In \emph{Proceedings of the Computer Vision and Pattern Recognition Conference}, pp.\ 20863--20873, 2025.
\bibitem[Ioffe \& Szegedy(2015)Ioffe and Szegedy]{batchnorm}
Sergey Ioffe and Christian Szegedy.
\newblock Batch normalization: Accelerating deep network training by reducing internal covariate shift.
\newblock In \emph{International conference on machine learning}, pp.\ 448--456. pmlr, 2015.
\bibitem[Isensee et~al.(2021)Isensee, Jaeger, Kohl, Petersen, and Maier-Hein]{nnunet}
Fabian Isensee, Paul~F Jaeger, Simon~AA Kohl, Jens Petersen, and Klaus~H Maier-Hein.
\newblock nnu-net: a self-configuring method for deep learning-based biomedical image segmentation.
\newblock \emph{Nature methods}, 18\penalty0 (2):\penalty0 203--211, 2021.
\bibitem[Kaittanis et~al.(2014)]{mri_therapy2}
Charalambos Kaittanis et~al.
\newblock Environment-responsive nanophores for therapy and treatment monitoring via molecular mri quenching.
\newblock \emph{Nature communications}, 5\penalty0 (1):\penalty0 3384, 2014.
\bibitem[Kapoor et~al.(2004)Kapoor, McCook, and Torok]{pet/ct2}
Vibhu Kapoor, Barry~M McCook, and Frank~S Torok.
\newblock An introduction to pet-ct imaging.
\newblock \emph{Radiographics}, 24\penalty0 (2):\penalty0 523--543, 2004.
\bibitem[Kramer et~al.(1989)Kramer, Locke, Byrd, and Daryabagi]{ct_therapy}
Lynn~D Kramer, GE~Locke, SE~Byrd, and JAFAR Daryabagi.
\newblock Cerebral cysticercosis: documentation of natural history with ct.
\newblock \emph{Radiology}, 171\penalty0 (2):\penalty0 459--462, 1989.
\bibitem[Kuang et~al.(2025)Kuang, Wang, Tan, Yang, Sun, Liu, Qiu, Zhang, Zhang, Yang, et~al.]{kuang2025lw}
Hulin Kuang, Yahui Wang, Xianzhen Tan, Jialin Yang, Jiarui Sun, Jin Liu, Wu~Qiu, Jingyang Zhang, Jiulou Zhang, Chunfeng Yang, et~al.
\newblock Lw-ctrans: A lightweight hybrid network of cnn and transformer for 3d medical image segmentation.
\newblock \emph{Medical Image Analysis}, 102:\penalty0 103545, 2025.
\bibitem[Li et~al.(2025)Li, Qin, Yuan, and Chen]{hcma_unet}
Haoxuan Li, Peiwu Qin, Xi~Yuan, and Zhenglin Chen.
\newblock Hcma-unet: A hybrid cnn-mamba unet with axial self-attention for efficient breast cancer segmentation.
\newblock \emph{arXiv preprint arXiv:2501.00751}, 2025.
\bibitem[Li et~al.(2017)Li, Wang, Liu, and Hou]{gram_math1}
Yanghao Li, Naiyan Wang, Jiaying Liu, and Xiaodi Hou.
\newblock Demystifying neural style transfer.
\newblock 2017.
\bibitem[Likhosherstov et~al.(2021)Likhosherstov, Choromanski, and Weller]{jl_attention}
Valerii Likhosherstov, Krzysztof Choromanski, and Adrian Weller.
\newblock On the expressive power of self-attention matrices.
\newblock \emph{arXiv preprint arXiv:2106.03764}, 2021.
\bibitem[Lindenstrauss \& Johnson(1984)Lindenstrauss and Johnson]{JL_Lemma}
W~Johnson~J Lindenstrauss and J~Johnson.
\newblock Extensions of lipschitz maps into a hilbert space.
\newblock \emph{Contemp. Math}, 26\penalty0 (189-206):\penalty0 2, 1984.
\bibitem[Liu et~al.(2025)Liu, Chen, Shi, Lu, Jian, Pan, Cai, Wang, Zhang, Li, et~al.]{liu2025does}
Che Liu, Yinda Chen, Haoyuan Shi, Jinpeng Lu, Bailiang Jian, Jiazhen Pan, Linghan Cai, Jiayi Wang, Yundi Zhang, Jun Li, et~al.
\newblock Does dinov3 set a new medical vision standard?
\newblock \emph{arXiv preprint arXiv:2509.06467}, 2025.
\bibitem[Liu et~al.(2024{\natexlab{a}})Liu, Bai, Torigian, Tong, and Udupa]{vsmtrans}
Tiange Liu, Qingze Bai, Drew~A Torigian, Yubing Tong, and Jayaram~K Udupa.
\newblock Vsmtrans: A hybrid paradigm integrating self-attention and convolution for 3d medical image segmentation.
\newblock \emph{Medical image analysis}, 98:\penalty0 103295, 2024{\natexlab{a}}.
\bibitem[Liu et~al.(2021)]{swint}
Ze~Liu et~al.
\newblock Swin transformer: Hierarchical vision transformer using shifted windows.
\newblock In \emph{Proceedings of the IEEE/CVF international conference on computer vision}, pp.\ 10012--10022, 2021.
\bibitem[Liu(2022)]{warmup_cosine}
Zhao Liu.
\newblock Super convergence cosine annealing with warm-up learning rate.
\newblock In \emph{CAIBDA 2022; 2nd International Conference on Artificial Intelligence, Big Data and Algorithms}, pp.\ 1--7, 2022.
\bibitem[Liu et~al.(2024{\natexlab{b}})]{kan}
Ziming Liu et~al.
\newblock Kan: Kolmogorov-arnold networks.
\newblock \emph{arXiv preprint arXiv:2404.19756}, 2024{\natexlab{b}}.
\bibitem[Loshchilov \& Hutter(2017)Loshchilov and Hutter]{adamw}
Ilya Loshchilov and Frank Hutter.
\newblock Decoupled weight decay regularization.
\newblock \emph{arXiv preprint arXiv:1711.05101}, 2017.
\bibitem[Lu et~al.(2019)Lu, Batra, Parikh, and Lee]{attn_text_image1}
Jiasen Lu, Dhruv Batra, Devi Parikh, and Stefan Lee.
\newblock Vilbert: Pretraining task-agnostic visiolinguistic representations for vision-and-language tasks.
\newblock \emph{Advances in neural information processing systems}, 32, 2019.
\bibitem[Ma et~al.(2018)Ma, Zhang, Zheng, and Sun]{shufflenetv2}
Ningning Ma, Xiangyu Zhang, Hai-Tao Zheng, and Jian Sun.
\newblock Shufflenet v2: Practical guidelines for efficient cnn architecture design.
\newblock In \emph{ECCV}, 2018.
\bibitem[Manara et~al.(2011)]{mri_therapy1}
Renzo Manara et~al.
\newblock Brain and spine mri features of hunter disease: frequency, natural evolution and response to therapy.
\newblock \emph{Journal of Inherited Metabolic Disease: Official Journal of the Society for the Study of Inborn Errors of Metabolism}, 34\penalty0 (3):\penalty0 763--780, 2011.
\bibitem[Mansour et~al.(2019)]{inductive_bias}
Tarek Mansour et~al.
\newblock \emph{Deep neural networks are lazy: on the inductive bias of deep learning}.
\newblock PhD thesis, Massachusetts Institute of Technology, 2019.
\bibitem[Menze et~al.(2014)]{mri1}
Bjoern~H Menze et~al.
\newblock The multimodal brain tumor image segmentation benchmark (brats).
\newblock \emph{IEEE transactions on medical imaging}, 34\penalty0 (10):\penalty0 1993--2024, 2014.
\bibitem[Milletari et~al.(2016)Milletari, Navab, and Ahmadi]{vnet}
Fausto Milletari, Nassir Navab, and Seyed-Ahmad Ahmadi.
\newblock V-net: Fully convolutional neural networks for volumetric medical image segmentation.
\newblock In \emph{2016 fourth international conference on 3D vision (3DV)}, pp.\ 565--571. Ieee, 2016.
\bibitem[Molchanov et~al.(2019)Molchanov, Mallya, Tyree, Frosio, and Kautz]{taylor_importance}
Pavlo Molchanov, Arun Mallya, Stephen Tyree, Iuri Frosio, and Jan Kautz.
\newblock Importance estimation for neural network pruning.
\newblock In \emph{Proceedings of the IEEE/CVF conference on computer vision and pattern recognition}, pp.\ 11264--11272, 2019.
\bibitem[Mu et~al.(2020)]{pet/ct1}
Wei Mu et~al.
\newblock Non-invasive decision support for nsclc treatment using pet/ct radiomics.
\newblock \emph{Nature communications}, 11\penalty0 (1):\penalty0 5228, 2020.
\bibitem[Muhammad et~al.(2025)Muhammad, Laaksonen, and Mihaylova]{muhammad2025towards}
Usman Muhammad, Jorma Laaksonen, and Lyudmila Mihaylova.
\newblock Towards lightweight hyperspectral image super-resolution with depthwise separable dilated convolutional network.
\newblock \emph{arXiv preprint arXiv:2505.00374}, 2025.
\bibitem[Oreiller et~al.(2022)]{Hecktor2022}
Valentin Oreiller et~al.
\newblock Head and neck tumor segmentation in pet/ct: the hecktor challenge.
\newblock \emph{Medical image analysis}, 77:\penalty0 102336, 2022.
\bibitem[Pang et~al.(2024)]{slimunetr}
Yan Pang et~al.
\newblock Slim unetr: Scale hybrid transformers to efficient 3d medical image segmentation under limited computational resources.
\newblock \emph{IEEE transactions on medical imaging}, 43\penalty0 (3):\penalty0 994--1005, 2024.
\bibitem[Peiris et~al.(2023)Peiris, Hayat, Chen, Egan, and Harandi]{peiris2023uncertainty}
Himashi Peiris, Munawar Hayat, Zhaolin Chen, Gary Egan, and Mehrtash Harandi.
\newblock Uncertainty-guided dual-views for semi-supervised volumetric medical image segmentation.
\newblock \emph{Nature Machine Intelligence}, 5\penalty0 (7):\penalty0 724--738, 2023.
\bibitem[Peng et~al.(2023)]{rwkv}
Bo~Peng et~al.
\newblock Rwkv: Reinventing rnns for the transformer era.
\newblock \emph{arXiv preprint arXiv:2305.13048}, 2023.
\bibitem[Perera et~al.(2024)Perera, Navard, and Yilmaz]{segformer3d}
Shehan Perera, Pouyan Navard, and Alper Yilmaz.
\newblock Segformer3d: an efficient transformer for 3d medical image segmentation.
\newblock In \emph{Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition}, pp.\ 4981--4988, 2024.
\bibitem[Qiu et~al.(2023)Qiu, Hu, Chen, Zeng, Hu, and Liu]{ds2f}
Zhongxi Qiu, Yan Hu, Xiaoshan Chen, Dan Zeng, Qingyong Hu, and Jiang Liu.
\newblock Rethinking dual-stream super-resolution semantic learning in medical image segmentation.
\newblock \emph{IEEE Transactions on Pattern Analysis and Machine Intelligence}, 46\penalty0 (1):\penalty0 451--464, 2023.
\bibitem[Rahman et~al.(2024)Rahman, Munir, and Marculescu]{emcad}
Md~Mostafijur Rahman, Mustafa Munir, and Radu Marculescu.
\newblock Emcad: Efficient multi-scale convolutional attention decoding for medical image segmentation.
\newblock In \emph{Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition}, pp.\ 11769--11779, 2024.
\bibitem[Roy et~al.(2023)]{mednext}
Saikat Roy et~al.
\newblock Mednext: transformer-driven scaling of convnets for medical image segmentation.
\newblock In \emph{International Conference on Medical Image Computing and Computer-Assisted Intervention}, pp.\ 405--415. Springer, 2023.
\bibitem[Rui et~al.(2025)Rui, Chen, Tang, Wang, Liu, Zhang, and Wang]{rui2025multi}
Shaohao Rui, Lingzhi Chen, Zhenyu Tang, Lilong Wang, Mianxin Liu, Shaoting Zhang, and Xiaosong Wang.
\newblock Multi-modal vision pre-training for medical image analysis.
\newblock In \emph{Proceedings of the Computer Vision and Pattern Recognition Conference}, pp.\ 5164--5174, 2025.
\bibitem[Sang et~al.(2021)Sang, Sun, Wang, Qi, and Li]{cogseg}
Yu~Sang, Jinguang Sun, Simiao Wang, Heng Qi, and Keqiu Li.
\newblock Super-resolution and infection edge detection co-guided learning for covid-19 ct segmentation.
\newblock In \emph{ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)}, pp.\ 1665--1669, 2021.
\newblock \doi{10.1109/ICASSP39728.2021.9414327}.
\bibitem[Shaker et~al.(2024)Shaker, Maaz, Rasheed, Khan, Yang, and Khan]{unetr++}
Abdelrahman~M Shaker, Muhammad Maaz, Hanoona Rasheed, Salman Khan, Ming-Hsuan Yang, and Fahad~Shahbaz Khan.
\newblock Unetr++: delving into efficient and accurate 3d medical image segmentation.
\newblock \emph{IEEE Transactions on Medical Imaging}, 2024.
\bibitem[Sim{\'e}oni et~al.(2025)Sim{\'e}oni, Vo, Seitzer, Baldassarre, Oquab, Jose, Khalidov, Szafraniec, Yi, Ramamonjisoa, et~al.]{dinov3}
Oriane Sim{\'e}oni, Huy~V Vo, Maximilian Seitzer, Federico Baldassarre, Maxime Oquab, Cijo Jose, Vasil Khalidov, Marc Szafraniec, Seungeun Yi, Micha{\"e}l Ramamonjisoa, et~al.
\newblock Dinov3.
\newblock \emph{arXiv preprint arXiv:2508.10104}, 2025.
\bibitem[Simpson et~al.(2019)]{MSD2019}
Amber~L Simpson et~al.
\newblock A large annotated medical image dataset for the development and evaluation of segmentation algorithms.
\newblock \emph{arXiv preprint arXiv:1902.09063}, 2019.
\bibitem[Su et~al.(2025)Su, Li, Xu, Fu, and Liu]{review_superresolution}
Hu~Su, Ying Li, Yifan Xu, Xiang Fu, and Song Liu.
\newblock A review of deep-learning-based super-resolution: From methods to applications.
\newblock \emph{Pattern Recognition}, 157:\penalty0 110935, 2025.
\bibitem[Sun et~al.(2020)Sun, Wang, Liu, Miller, Efros, and Hardt]{multi_task_theory}
Yu~Sun, Xiaolong Wang, Zhuang Liu, John Miller, Alexei Efros, and Moritz Hardt.
\newblock Test-time training with self-supervision for generalization under distribution shifts.
\newblock In \emph{International conference on machine learning}, pp.\ 9229--9248. PMLR, 2020.
\bibitem[Ulyanov et~al.(2016)Ulyanov, Vedaldi, and Lempitsky]{instancenorm}
Dmitry Ulyanov, Andrea Vedaldi, and Victor Lempitsky.
\newblock Instance normalization: The missing ingredient for fast stylization.
\newblock \emph{arXiv preprint arXiv:1607.08022}, 2016.
\bibitem[Vaswani et~al.(2017)Vaswani, Shazeer, Parmar, Uszkoreit, Jones, Gomez, Kaiser, and Polosukhin]{attention_is_all_you_need}
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan~N Gomez, {\L}ukasz Kaiser, and Illia Polosukhin.
\newblock Attention is all you need.
\newblock \emph{Advances in neural information processing systems}, 30, 2017.
\bibitem[Wald et~al.(2025)Wald, Ulrich, Lukyanenko, Goncharov, Paderno, Miller, Maerkisch, Jaeger, and Maier-Hein]{wald2025revisiting}
Tassilo Wald, Constantin Ulrich, Stanislav Lukyanenko, Andrei Goncharov, Alberto Paderno, Maximilian Miller, Leander Maerkisch, Paul Jaeger, and Klaus Maier-Hein.
\newblock Revisiting mae pre-training for 3d medical image segmentation.
\newblock In \emph{Proceedings of the Computer Vision and Pattern Recognition Conference}, pp.\ 5186--5196, 2025.
\bibitem[Wang et~al.(2025{\natexlab{a}})Wang, Guo, Ye, Deng, Cheng, Li, Chen, Su, Huang, Shen, et~al.]{sam_med3d}
Haoyu Wang, Sizheng Guo, Jin Ye, Zhongying Deng, Junlong Cheng, Tianbin Li, Jianpin Chen, Yanzhou Su, Ziyan Huang, Yiqing Shen, et~al.
\newblock Sam-med3d: A vision foundation model for general-purpose segmentation on volumetric medical images.
\newblock \emph{IEEE Transactions on Neural Networks and Learning Systems}, 2025{\natexlab{a}}.
\bibitem[Wang et~al.(2021)]{pfseg}
Hongyi Wang et~al.
\newblock Patch-free 3d medical image segmentation driven by super-resolution technique and self-supervised guidance.
\newblock In \emph{Medical Image Computing and Computer Assisted Intervention--MICCAI 2021: 24th International Conference, Strasbourg, France, September 27--October 1, 2021, Proceedings, Part I 24}, pp.\ 131--141. Springer, 2021.
\bibitem[Wang et~al.(2025{\natexlab{b}})]{masktwins}
Jiawen Wang et~al.
\newblock Masktwins: Dual-form complementary masking for domain-adaptive image segmentation.
\newblock In \emph{Forty-second International Conference on Machine Learning}, 2025{\natexlab{b}}.
\bibitem[Wang \& Hong(2023)Wang and Hong]{a2fseg}
Zirui Wang and Yi~Hong.
\newblock A2fseg: Adaptive multi-modal fusion network for medical image segmentation.
\newblock In \emph{International Conference on Medical Image Computing and Computer-Assisted Intervention}, pp.\ 673--681. Springer, 2023.
\bibitem[Wu et~al.(2025)Wu, Zhang, Zhang, Hui, Wang, and Xie]{wu2025towards}
Chaoyi Wu, Xiaoman Zhang, Ya~Zhang, Hui Hui, Yanfeng Wang, and Weidi Xie.
\newblock Towards generalist foundation model for radiology by leveraging web-scale 2d\&3d medical data.
\newblock \emph{Nature Communications}, 16\penalty0 (1):\penalty0 7866, 2025.
\bibitem[Xing et~al.(2022)Xing, Yu, Wan, Han, and Zhu]{nestedformer}
Zhaohu Xing, Lequan Yu, Liang Wan, Tong Han, and Lei Zhu.
\newblock Nestedformer: Nested modality-aware transformer for brain tumor segmentation.
\newblock In \emph{International conference on medical image computing and computer-assisted intervention}, pp.\ 140--150. Springer, 2022.
\bibitem[Xing et~al.(2025)Xing, Ye, Yang, Cai, Gai, Wu, Gao, and Zhu]{xing2025segmamba}
Zhaohu Xing, Tian Ye, Yijun Yang, Du~Cai, Baowen Gai, Xiao-Jian Wu, Feng Gao, and Lei Zhu.
\newblock Segmamba-v2: Long-range sequential modeling mamba for general 3d medical image segmentation.
\newblock \emph{IEEE Transactions on Medical Imaging}, 2025.
\bibitem[Ye et~al.(2025)]{u_rwkv}
Hongbo Ye et~al.
\newblock U-rwkv: Lightweight medical image segmentation with direction-adaptive rwkv.
\newblock \emph{arXiv preprint arXiv:2507.11415}, 2025.
\bibitem[Yu et~al.(2025{\natexlab{a}})Yu, Zhou, Yuan, Liang, Shcherbakov, and Zhang]{yu20253d}
Bin Yu, Quan Zhou, Li~Yuan, Huageng Liang, Pavel Shcherbakov, and Xuming Zhang.
\newblock 3d medical image segmentation using the serial--parallel convolutional neural network and transformer based on cross-window self-attention.
\newblock \emph{CAAI Transactions on Intelligence Technology}, 10\penalty0 (2):\penalty0 337--348, 2025{\natexlab{a}}.
\bibitem[Yu et~al.(2025{\natexlab{b}})Yu, Cao, Liu, and Jiang]{superlightnet}
Feng Yu, Jiacheng Cao, Li~Liu, and Minghua Jiang.
\newblock Superlightnet: Lightweight parameter aggregation network for multimodal brain tumor segmentation.
\newblock In \emph{Proceedings of the Computer Vision and Pattern Recognition Conference}, pp.\ 5197--5206, 2025{\natexlab{b}}.
\bibitem[Zhang et~al.(2020)Zhang, Yu, Chen, Shi, Bao, and Ma]{auxiliary}
Linfeng Zhang, Muzhou Yu, Tong Chen, Zuoqiang Shi, Chenglong Bao, and Kaisheng Ma.
\newblock Auxiliary training: Towards accurate and robust models.
\newblock In \emph{Proceedings of the IEEE/CVF conference on computer vision and pattern recognition}, pp.\ 372--381, 2020.
\bibitem[Zheng et~al.(2025)Zheng, Ye, Yang, Yu, Li, Gao, and Zhao]{zheng2025asymmetric}
Shenhai Zheng, Xin Ye, Chaohui Yang, Lei Yu, Weisheng Li, Xinbo Gao, and Yue Zhao.
\newblock Asymmetric adaptive heterogeneous network for multi-modality medical image segmentation.
\newblock \emph{IEEE Transactions on Medical Imaging}, 2025.
\bibitem[Zhu et~al.(2021)Zhu, Ji, Zhu, Gan, Wu, and Yan]{texture}
Lanyun Zhu, Deyi Ji, Shiping Zhu, Weihao Gan, Wei Wu, and Junjie Yan.
\newblock Learning statistical texture for semantic segmentation.
\newblock In \emph{Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition}, pp.\ 12537--12546, 2021.
\bibitem[Zou et~al.(2025)Zou, Liu, Chen, Wang, Hu, Xing, and Qin]{zou2025mmr}
Jing Zou, Lanqing Liu, Qi~Chen, Shujun Wang, Zhanli Hu, Xiaohan Xing, and Jing Qin.
\newblock Mmr-mamba: Multi-modal mri reconstruction with mamba and spatial-frequency information fusion.
\newblock \emph{Medical Image Analysis}, 102:\penalty0 103549, 2025.
\end{thebibliography}