SkelMamba: A State Space Model for Efficient Skeleton Action Recognition of Neurological Disorders
Extracted source text · arXiv 2411.19544
% CVPR 2024 Paper Template; see https://github.com/cvpr-org/author-kit
\documentclass[10pt,twocolumn,letterpaper]{article}
\usepackage{titlesec}
%%%%%%%%% PAPER TYPE - PLEASE UPDATE FOR FINAL VERSION
\usepackage{cvpr} % To produce the CAMERA-READY version
%\usepackage[review]{cvpr} % To produce the REVIEW version
% \usepackage[pagenumbers]{cvpr} % To force page numbers, e.g. for an arXiv version
\usepackage{textcomp}
\usepackage{stfloats}
\usepackage{url}
\usepackage{verbatim}
\usepackage{graphicx}
\usepackage{cite}
\usepackage{amssymb}
\usepackage{amsmath,amsfonts}
\usepackage{pifont}
\usepackage{multirow}
\usepackage{booktabs}
\usepackage{float}
\usepackage{bm}
\usepackage{gensymb}
\usepackage{comment}
\usepackage{tabularx}
\usepackage{mathtools}
\usepackage{pifont}
\newcommand{\tick}{\ding{51}} % Tick
\newcommand{\cross}{\ding{55}} % Cross
\usepackage[table]{xcolor}
\definecolor{lightgray}{gray}{0.8} % Light gray color
\newcommand{\thinline}{\noalign{\vskip 0.1mm\color{lightgray}\hrule height 0.1mm}} % Thin line
\newcolumntype{Y}{>{\centering\arraybackslash}X}
\newcommand{\compfigfont}{\fontsize{5pt}{7pt}\selectfont}
\setlength{\heavyrulewidth}{0.15em} % Width of toprule and bottomrule
\setlength{\lightrulewidth}{0.1em} % Width of midrule
\setlength{\cmidrulewidth}{0.05em} % Default width of cmidrule
% Import additional packages in the preamble file, before hyperref
%\input{preamble}
% It is strongly recommended to use hyperref, especially for the review version.
% hyperref with option pagebackref eases the reviewers' job.
% Please disable hyperref *only* if you encounter grave issues,
% e.g. with the file validation for the camera-ready version.
%
% If you comment hyperref and then uncomment it, you should delete *.aux before re-running LaTeX.
% (Or just hit 'q' on the first LaTeX run, let it finish, and you should be clear).
\definecolor{cvprblue}{rgb}{0.21,0.49,0.74}
\usepackage[pagebackref,breaklinks,colorlinks,citecolor=cvprblue]{hyperref}
\graphicspath{{./figures/}}
\usepackage{balance}
\usepackage{xspace}
\usepackage{xstring}
\newcommand{\codestyle}[1]{\texttt{#1}\xspace}
\def\ln{\codestyle{LN}}
\def\tsmb{\codestyle{TSMB}}
\def\pgmb{\codestyle{PGMB}}
\def\pgml{\codestyle{PGM}}
\def\tdown{\codestyle{TDown}}
\def\mysplit{\codestyle{Split}}
\def\myconcat{\codestyle{Concat}}
\def\linear{\codestyle{Linear}}
\def\tconv{\codestyle{TemporalConv}}
\def\gconv{\codestyle{SpatialConv}}
\def\bn{\codestyle{BN}}
\def\ffn{\codestyle{FFN}}
\def\conv{\codestyle{Conv}}
\def\convoned{\codestyle{Conv1D}}
\def\convtwod{\codestyle{Conv2D}}
\def\attn{\codestyle{Attention}}
\def\gelu{\codestyle{GELU}}
\def\ssm{\codestyle{SSM}}
\def\mycssm{\codestyle{C2DSSM}}
\def\pool{\codestyle{Pool}}
\def\yhat{\mathbf{\hat{y}}}
\def\modality{\mathbb{M}}
\def\modalityj{\modality_j}
\def\modalityjb{\modality_{jb}}
\def\modalityjbm{\modality_{jbm}}
\def\secondbestcell{\cellcolor{yellow!50}}
\def\bestcell{\cellcolor{green!25}}
% Remove the space above the \subsubsection
\titlespacing*{\subsubsection}{\parindent}{0pt}{0pt}
% Redefine the \subsubsection to be inline and remove its numbering
\titleformat{\subsubsection}[runin]% 'runin' puts it in the same paragraph
{\normalfont\normalsize\bfseries}% The style of the \subsubsection
{}% No label (numbering)
{0pt}% No space between the label and title body
{}% No prefix code
[] %
%%%%%%%%% PAPER ID - PLEASE UPDATE
\def\paperID{7217} % *** Enter the Paper ID here
\def\confName{CVPR}
\def\confYear{2025}
%%%%%%%%% TITLE - PLEASE UPDATE
\title{
\begin{tabularx}{\linewidth}{l X r}
\raisebox{-2em}[0pt][0pt]{\hspace{2em}\includegraphics[height=3em]{figures/mamba_skel_icon.png}} &
\centering SkelMamba: A State Space Model for Efficient Skeleton Action Recognition of Neurological Disorders &
\raisebox{-2em}[0pt][0pt]{\includegraphics[height=3em]{figures/mamba_skel_icon.png}}
\\
\end{tabularx}
}
%\title{\S}
%%%%%%%%% AUTHORS - PLEASE UPDATE
\author{Niki Martinel\\
University of Udine\\
{\tt\small niki.martinel@uniud.it}
% For a paper whose authors are all at the same institution,
% omit the following lines up until the closing ``}''.
% Additional authors and addresses can be added with ``\and'',
% just like the second author.
% To save space, use either the email address or home page, not both
\and
Mariano Serrao\\
Sapienza Università di Roma\\
{\tt\small mariano.serrao@uniroma1.it}
\and
Christian Micheloni\\
University of Udine\\
{\tt\small christian.micheloni@uniud.it}
}
\begin{document}
\maketitle
\input{sec/0_abstract}
\input{sec/1_intro}
\input{sec/2_related_work}
\input{sec/3_method}
\input{sec/4_exp}
\input{sec/5_ablation}
\input{sec/6_conclusion}
{
\small
\bibliographystyle{ieeenat_fullname}
\bibliography{main}
}
\end{document}
\begin{abstract}
We introduce a novel state-space model (SSM)-based framework for skeleton-based human action recognition, with an anatomically-guided architecture that improves state-of-the-art performance in both clinical diagnostics and general action recognition tasks.
Our approach decomposes skeletal motion analysis into spatial, temporal, and spatio-temporal streams, using channel partitioning to capture distinct movement characteristics efficiently.
By implementing a structured, multi-directional scanning strategy within SSMs, our model captures local joint interactions and global motion patterns across multiple anatomical body parts. This anatomically-aware decomposition enhances the ability to identify subtle motion patterns critical in medical diagnosis, such as gait anomalies associated with neurological conditions.
On public action recognition benchmarks,~\ie, NTU RGB+D, NTU RGB+D 120, and NW-UCLA, our model outperforms current state-of-the-art methods, achieving accuracy improvements up to 3.2\% with lower computational complexity than previous leading transformer-based models.
We also introduce a novel medical dataset for motion-based patient neurological disorder analysis to validate our method’s potential in automated disease diagnosis.
\end{abstract}
\section{Introduction}
\label{sec:intro}
Human action recognition is the task of classifying actions based on human movements.
The problem is often tackled by leveraging the rich contextual features in RGB videos --at the cost of exposing information about people's identities.
Skeleton-based action recognition has emerged as a privacy-preserving alternative for sensitive applications, from patient monitoring and physical therapy to assisted living environments.
Skeletal 3D joint representations are compact and robust to environmental conditions (\eg, background clutter and light variations) yet their sparse nature makes this task inherently challenging.
In the medical domain, precisely capturing the dynamic spatio-temporal relationships between joints is of paramount importance for precise analysis of subtle movements indicative of various diseases.
For example, analyzing a patient's gait can provide insights into neurological disorders, musculoskeletal abnormalities, and other health conditions.
The skeleton joints and their connections (\ie, bones) correspond to the vertices and edges of a graph structure.
Our community has recently proposed skeleton action recognition methods that model the spatial and temporal dependencies among skeletal joints via Graph Convolution Networks (GCN) or transformer-like architectures.
GCN-based methods introduced adaptive graph structures (\eg,~\cite{shi2019dgnn, chi2022infogcn, shi2019agcn}), specialized joint encodings (\eg,~\cite{lin2023actionlet, cai2021jologcn}), and explored multiple modalities (\eg,~\cite{liu2024mmcolearning}) for learning robust representations (\eg,~\cite{hou2022graphmae,zhou2023,lin2023actionlet}).
Transformer-based methods tackle the long-range dependencies in skeletal data that GCN-based methods often struggle with.
Existing methods model the skeletal spatio-temporal relationship between physically neighboring and distant joints/frames using the self-attention mechanism (\eg,~\cite{zhou2022hyperformer, do2024skateformer}) in one-shot settings (\eg,~\cite{zhu2023adaptive}) or for joint training across different action tasks and datasets (\eg,~\cite{duan2023skeletr}).
Transformer-like architectures are computationally demanding and GCN-based methods struggle to model the relation of physically distant joints --captured through the direct propagation of information between physically connected joints.
This motivates the introduction of our novel state-space model (SSM)-based architecture, which models all joint relationships with an efficient spatial-temporal scanning strategy --designed to analyze skeletal data for patient disease recognition.
Our novel approach introduces a structured decomposition of skeletal motion data operating across three complementary dimensions. Given an input sequence, we first partition its channel representations into specialized groups for spatial, temporal, and spatio-temporal analysis.
The spatial and temporal streams capture local patterns and short-range frame-wise transitions, while the spatio-temporal stream introduces State Space Models (SSMs) for complex motion modeling.
Neurological disorders impact distinct body parts during locomotion, resulting in disease-specific motion patterns.
Within the spatio-temporal stream, we further partition the input according to anatomically meaningful body parts (\eg, legs, torso, arms) and their key interactions (\eg, arms-legs coordination) that are analyzed by separate SSMs.
Each SSM features our novel four-way scanning strategy that splits each anatomical group into four channel subgroups.
A specific scanning direction is applied to a specific subgroup.
This enables efficient parallel processing --while reducing the computational demands of our model-- and allows us to collectively analyze the motion patterns across both space and time: from temporal to spatial domain, spatial to temporal domain, and their respective inverse directions.
This multi-directional scanning enables comprehensive capture of both local joint relationships and global motion patterns while being computationally efficient.
This new anatomically-aware architecture proves particularly effective for automated medical diagnosis, where subtle motion abnormalities often manifest through complex interactions between different body parts over time.
Nevertheless, our model remains generic and demonstrates significant improvements over state-of-the-art results on existing challenging action recognition datasets, showcasing its versatility and robustness.
Our contribution is threefold:
\begin{itemize}
\item We propose a novel multi-stream architecture leveraging SSMs that effectively decomposes motion analysis into spatial, temporal, and spatio-temporal streams through channel partitioning, enabling efficient parallel processing of distinct motion characteristics.
\item We introduce an anatomically-aware partitioning scheme that guides the SSM analysis based on meaningful body parts and their interactions, capturing both local joint dynamics and complex cross-body motion patterns crucial for medical diagnosis.
\item We develop a channel-split scanning mechanism where input features are partitioned into four subgroups, each processed by direction-specific SSMs. This approach enables comprehensive multi-directional motion analysis while maintaining computational efficiency through reduced channel dimensionality.
\end{itemize}
Through extensive experiments on both medical diagnosis tasks (we introduce a dataset for the analysis of patient walking styles to assist in automated neurological disease diagnosis) and standard action recognition benchmarks, we demonstrate that our method achieves state-of-the-art performance while being very computationally efficient.
% \section{Related Work}
% \subsection{GCN-based approaches}
% Graph Convolutional Networks (GCNs) have been extensively leveraged for skeleton-based action recognition due to their strength in capturing the complex spatiotemporal dependencies in human joint data. Yan et al. \cite{yan2018spatial} pioneered the application of GCNs in this domain by introducing the Spatial-Temporal Graph Convolutional Network (ST-GCN), which models the skeleton data as spatiotemporal graphs to extract action patterns. Following this, a plethora of methods have emerged to enhance the GCN framework. For instance, Shi et al. \cite{shi2019two} proposed an adaptive GCN framework integrating self-attention mechanisms to dynamically learn the graph topology, while Peng et al. \cite{peng2020learning} introduced a two-stream architecture that integrates joint and bone information for better feature representation.
% Deformable GCN (DeGCN) by Myung et al. \cite{myung2020degcn} addresses the inflexibility in traditional GCNs by allowing adaptive graph structure learning. Ye et al. \cite{ye2020dynamic} further enhanced the topology learning aspect with their Dynamic GCN, which is context-enriched for better handling diverse action recognition scenarios. Other notable works include Shi et al.’s MS-G3D \cite{shi2020skeleton}, which improves multi-scale modeling of spatiotemporal graphs, and Zhang et al.'s AGCN \cite{zhang2020context}, integrating both contextual and edge features into the learning process.
% Recent advancements look into the computational efficiency and scalability of GCN-based models. Cheng et al. \cite{cheng2020shift} proposed Shift-GCN which drastically reduces computational complexity while maintaining high accuracy by shifting the input graphs. Yang et al. \cite{yang2020efficient} introduced a compact yet effective structure through ST-GCN++, leveraging graph coarsening for scaling to higher resolution inputs while minimizing the computational burden.
% Beyond performance optimization, multi-modality approaches have also gained traction. For example, Lin et al. \cite{lin2020multi} integrate multiple sensor modalities into the GCN framework, providing a more robust recognition system. Wang and Wang \cite{wang2017modeling} explored spatiotemporal configurations using two-stream RNNs and integrated GCNs for enriched action feature learning.
% Despite these advancements, existing GCNs often struggle with a fixed and manually defined graph topology, which may not be optimal for all action samples. Our approach diverges by deploying state-space models (SSMs) that dynamically partition the latent representations, leveraging multiple sub-models to capture both local and global joint interactions efficiently. This flexibility allows capturing subtle gait abnormalities in medical diagnosis scenarios, a critical aspect often overlooked by traditional GCN methods.
% \subsection{Transformer-based approaches}
% Transformers have emerged as a robust alternative to RNN and GCN-based models for skeleton action recognition. These architectures excel at modeling long-range dependencies and capturing the spatiotemporal dynamics of skeletal data.
% The pioneering work of Vaswani et al. \cite{vaswani2017attention} introduced the transformer model, which has since been adapted for various applications in computer vision. ST-TR by Plizzari et al. \cite{plizzari2021spatial} was among the first to apply transformers specifically for skeleton data, demonstrating their efficacy in capturing joint correlations via spatial and temporal attention mechanisms. Follow-up works like PoseFormer \cite{poseformer2021} and ActionFormer \cite{wei2021actionformer} refine these ideas by integrating multi-scale feature extraction and leveraging temporal consistency, achieving significant performance improvements.
% Wu et al. \cite{wu2022spatial} presented a transformer with spatial-temporal partitioning capabilities that allow focus on key frames and joints, reducing computational overhead while maintaining performance. Similarly, Kim et al. \cite{kim2022global} introduced a Global-Local Motion Transformer that combines global attention with local joint dynamics, capturing intricate human motions more effectively.
% Researchers have also explored domain-specific augmentations to transformers. For example, FreqMixFormer by Ranasinghe et al. \cite{ranasinghe2022frequency} incorporates frequency domain features to better differentiate between subtle motion patterns, while SkateFormer \cite{skateformer2022} optimizes computational efficiency by balancing parameter and FLOP counts with discriminative ability.
% Xu et al. \cite{xu2023multimodal} extended transformer models' utility by integrating multiple modalities, such as RGB, depth, and skeleton data, for a more robust action recognition system. This trend is also evident in SkeleTR by Duan et al. \cite{duan2021skeletr}, which supports multiple interaction tasks, proving its versatility in real-world scenarios.
% In the realm of self-supervised learning, skeleton-focused transformers are gaining importance. H-transformer \cite{kim2021htransformer} is a notable example, employing self-supervised pretraining by predicting joint velocities to enhance performance on downstream tasks.
% To address the heavy computational costs associated with transformer architectures, recent innovations have proposed lightweight designs. Liu et al. \cite{liu2022lightweight} introduced a skeleton transformer with reduced attention map sizes via pooling techniques. Additionally, IGFormer by Kim et al. \cite{kim2023interactiongraph} emphasizes computational efficiency coupled with strong interaction recognition capabilities.
% Despite these advances, current transformer-based models often neglect the fine-tuning of local joint interactions, which can be crucial for capturing subtle abnormalities in skeletal movements. Our proposed method partitions the latent representations across channel dimensions, allowing a set of SSMs to concurrently process localized and global joint interactions, optimizing both computational efficiency and diagnostic accuracy for detecting medical conditions through gait analysis.
% \subsection{SSM approaches}
% State-space models (SSMs) have proven to be essential in effectively modeling temporal dynamics and capturing structured data across various domains. When applied to skeleton-based action recognition, SSMs demonstrate significant improvements by capturing the intricate dependencies within skeletal data.
% Several works explore the applications of SSMs in the realm of skeleton action recognition. For instance, Liu et al. \cite{liu2019multi} implemented a self-supervised learning approach using SSMs for temporal dependency modeling in multi-task frameworks, which enhanced skeleton action recognition performance. Similarly, the SkeletonMAE model \cite{liu2022skeletonmae} utilized a graph-based masked autoencoder embedding SSM principles to learn effective skeleton sequence representations, yielding state-of-the-art results.
% Other notable research includes Schmidt et al. \cite{schmidt2020learning}, who integrated latent space models with recurrent neural networks (RNNs) to identify and segment human actions by leveraging latent state inference. This methodology has shown that understanding latent states provides superior representation for capturing the spatiotemporal patterns in skeleton data. Further, Chen et al. \cite{chen2022dynamic} proposed dynamic SSMs for adaptive spatio-temporal modeling, further enhancing action recognition accuracy.
% SSMs have also been adapted in other domains, highlighting their versatility. For example, state-space models have been applied to dynamic gesture recognition by Hori et al. \cite{hori2020early}, where they captured the dynamic nature of gestures with higher accuracy and efficiency. Similarly, the work on time-series forecasting by Rangapuram et al. \cite{rangapuram2018deep} utilized deep SSMs, yielding notable improvements in predictive accuracy across various datasets.
% Further insights came from the work on traffic prediction by Polson and Sokolov \cite{polson2017deep}, where state-space models were used to handle non-linearities and uncertainties in traffic flow data. This application underscored the robustness of SSMs in dealing with complex real-world problems.
% The combination of SSMs with graph-based methods has been beneficial as well. Tang et al. \cite{tang2018deep} integrated SSMs with graph neural networks to achieve state-of-the-art results in skeleton action recognition. They illustrated significant improvements by modeling the temporal evolution and spatial relationships between joints.
% In medical diagnosis, SSMs have been adapted for physiological signal processing, as demonstrated by the work of Sun et al. \cite{sun2018physiological}, where they captured the temporal dynamics of ECG signals to enhance disease detection accuracy.
% The effectiveness of SSMs extends to natural language processing (NLP) applications as well. Kim et al. \cite{kim2019semi} utilized SSMs for sequence labeling tasks, showcasing their ability to model sequential dependencies in text data, resulting in superior performance on NLP benchmarks.
% Despite the widespread success of SSMs in capturing temporal and spatial dependencies, challenges remain. Our innovative approach introduces partitioning latent representations across channels, allowing multiple SSMs to process local and global joint interactions concurrently. This method optimizes computational efficiency and diagnostic accuracy, especially in medical applications where detecting subtle gait abnormalities is crucial.
% \subsection{Skeleton Action Recognition for Automated Diagnosis}
% The application of skeleton-based action recognition in the medical field, particularly for automated diagnosis of diseases such as Parkinson’s disease (PD), has seen significant advancements. These efforts focus on leveraging high-precision human pose estimation algorithms to identify subtle movement abnormalities indicative of various medical conditions.
% Early detection of Parkinson's disease through gait analysis has gained traction. For example, the twin-tower transformer network proposed by Li et al. \cite{LiTwin2023} transforms skeletal coordinate sequences into multivariate time series and classifies them to determine the presence of Parkinson’s disease, showcasing a practical approach to integrating different time series classification algorithms. The work by Naseem et al. \cite{Naseem2024} also combines foot pressure and skeleton data through early and late fusion techniques to enhance pathological gait classification.
% The study by Guo et al. \cite{Guo2022} integrates graph convolutional networks (GCNs) for fine-grained action recognition, specifically targeting Parkinsonian Freezing-of-Gait (FOG) assessment.
% They leverage the spatiotemporal relationships among key joints to enhance model accuracy, a method further enhanced by weighted matrices designed to capture topological properties of the human skeleton, as proposed by Wen et al. \cite{Wen2019} and later by Shi et al. \cite{Shi2019} in their two-stream adaptive GCNs.
% Moreover, benchmarking efforts by Vida Adeli et al. \cite{Adeli2022} highlight the potential of skeleton-based motion encoder models for clinical applications, establishing benchmarks for analyzing skeletal motion in PD severity estimation. This emphasizes not only the clinical utility of these models but also the necessity to customize them further to capture disease-specific features effectively.
% Additionally, Nguyen et al. \cite{Nguyen2018} proposed methods that utilized 3D skeleton data to predict gait abnormality indexes, further exemplifying the integration of skeletal data into gait analysis for abnormal motion recognition. These methods have been fine-tuned to capture specific motion patterns linked to disease diagnosis.
% While the existing state-of-the-art approaches provide robust frameworks for disease detection and staging, many of these methods rely either on complex preprocessing steps or heavily on large-scale labeled datasets, which can be restrictive. Our proposed approach introduces a novel model architecture based on state-space models (SSMs) designed specifically to analyze walking skeletal data for patient disease recognition. By partitioning latent representations across channel dimensions and enabling independent SSMs to model distinct spatiotemporal relationships, our model excels in capturing both local and global interactions within skeletal data. This advances the accuracy and reliability of detecting subtle gait abnormalities crucial for medical diagnoses, ensuring efficient and scalable performance. Our method also demonstrates significant improvements over state-of-the-art results on existing challenging action recognition datasets, showcasing its versatility and robustness in both general and medical application domains.
\section{Related Work}
\subsubsection{Graph Convolutional Networks (GCNs)}~have been initially explored for skeleton-based action recognition in~\cite{yan2018stgcn}.
The seminal work introduced the Spatial-Temporal Graph Convolutional Network (ST-GCN) framework for modeling human joints with spatiotemporal graph structures.
Recent architectural innovations have evolved along multiple dimensions, from joint-bone fusion networks \cite{peng2020gnnsearch} and multi-scale feature extraction \cite{shi2020msagcn} to adaptive graph topology learning \cite{shi2019agcn}.
Context-aware architectures \cite{zhang2020contextawaregcn, chen2021ctrgcn} and dual-stream temporal models \cite{shi2020dstanet} have enhanced feature extraction capabilities, while adaptive graph structures \cite{myung2024degcn, ye2020dyngcn} and efficient convolutions \cite{cheng2020shiftgcn, duan2022pyskl} have optimized computational overhead.
These developments, complemented by multi-modal integration approaches \cite{liu2024mmcolearning}, have significantly advanced the field's ability to capture rich motion characteristics while maintaining robustness across varying conditions.
GCN-based approaches rely on local graph operations and predefined adjacency matrices, limiting their ability to capture long-range dependencies and dynamic motion patterns.
Our approach diverges by introducing state-space models (SSMs) for dynamic latent space partitioning, efficiently capturing local and global interactions.
\subsubsection{Transformers}~are a powerful alternative architecture for skeleton-based action recognition, primarily due to their capacity to model complex joints and temporal motion patterns through the self-attention mechanism \cite{vaswani2017attention}.
Spatial-temporal attention frameworks \cite{plizzari2021sttr, duan2023skeletr} enable joint modeling of structural and dynamic information, while global-local attention mechanisms \cite{kim2022globallocaltransformer} selectively focus on key motion patterns across different temporal scales.
%Domain-specific innovations have further enhanced transformer capabilities.
Frequency-aware architectures \cite{wu2024freqaware} improve data efficiency through spectral augmentation, while specialized designs \cite{do2024skateformer} optimize performance for specific motion types.
Multi-modal approaches \cite{wang20233mformer} incorporate complementary sensor data to enhance robustness across varying conditions.
Self-supervised pretraining strategies \cite{zhou2022hyperformer} leverage unlabeled data for improved representation learning, while efficient architectural designs \cite{pang2022igformer, oraki2024lortsar} maintain high accuracy with reduced computational demands.
Despite these advances, transformer-based methods face inherent limitations due to their quadratic computational complexity in modeling pairwise attention, motivating our exploration of more efficient architectures for real-time applications.
\subsubsection{State-space models (SSMs)}~offer an efficient approach to sequence modeling with long-range dependencies. Initial work on linear state-space layers~\cite{gu2021combining} laid the foundation for S4~\cite{gu2022s4}, while subsequent variants~\cite{gupta2022diagonalssm,smith2023simplifiedssm,hasani2022liquidssm} demonstrated comparable performance with simplified architectures.
Mamba~\cite{guo2023ssmmamba} addressed content-based reasoning limitations through input-dependent parameters~\cite{fu2022hungry}.
For vision tasks, existing SSM adaptations~\cite{nguyen2022s4nd,ma2024umamba,archit2024vimunet,patro2024simba} primarily follow fixed unidirectional scanning patterns.
Video-based methods~\cite{yang2024vivim,zhang2024motionmamba,chen2024videomamba} process temporal information sequentially after spatial encoding, while 3D approaches~\cite{zhang2024voxelmamba,liang2024pointmamba,shaker2024groupmamba} rely on predetermined serialization strategies.
Recent works have further expanded SSMs to image restoration~\cite{deng2024cumamba} and speech processing~\cite{liu2024spmamba}, with surveys~\cite{xu2024mambasurvey} providing comprehensive overviews.
While SSMs offer flexibility, their adaptation to skeleton-based action recognition remains challenging.
The recent work in~\cite{chaudhuri2024simba} addressed the problem with a temporal-driven one-direction scanning strategy applied on the top of GCN latent space embeddings.
In contrast, we enable simultaneous four-way scanning (spatial-temporal, temporal-spatial, in both forward/backward directions) through channel grouping.
Experimental comparisons show that this efficient multi-directional processing captures richer motion dependencies yielding improved skeleton-based analysis.
\begin{figure*}[!t]
\centering
\includegraphics[width=1\linewidth]{skelemamba.png}
\caption{The overall framework of our proposed SkelMamba architecture.}
\label{fig:framework}
\end{figure*}
\subsubsection{Automated diagnosis of neurological disorders}~has made significant progress through multi-modal approaches and advanced architectures.
Recent methods have demonstrated success by integrating skeleton data with foot pressure information for Parkinson's Disease (PD) assessment \cite{naseem2024gaitfootpressureskeleton}, while graph-based networks enhanced with causality mechanisms have improved Freezing of Gait detection \cite{guo2024causalitygcn}.
Clinical findings have shown that fine-tuned motion encoders effectively capture pathological gait patterns \cite{adeli2024benchmarkingskeletonparkinson}, complemented by spatiotemporal architectures for PD recognition \cite{zhang2023wmstgcn}.
Vision-based ensemble discriminate between PD and knee osteoarthritis gaits \cite{kour2024koanmpddataset}, building upon established clinical characterizations of disorder-specific gait patterns \cite{10.1093/brain/122.7.1349, Serrao2016}.
More recently, transformer architectures have demonstrated promising results in early PD detection \cite{ma2024twintower}.
These advances underscore the growing potential of computer vision for objective clinical gait assessment \cite{guo2022parkinsongaitsurvey}.
These existing approaches often hinge on complex preprocessing/modeling architecture or raise privacy concerns through video analysis.
Our model operates directly on skeletal data and captures distinct spatiotemporal dynamics in walking patterns without requiring extensive computational resources or compromising patient privacy, advancing the state-of-the-art in skeleton-based action recognition.
\section{SkelMamba}
\label{sec:method}
The SkelMamba architecture is illustrated in~\figurename~\ref{fig:framework}.
A skeleton sequence is denoted as $\mathbf{X} \in \mathbb{R}^{T \times V \times C}$, where $T$ is the sequence length, $V$ is the number of joints per frame, and $C$ represents the joint coordinates.
%To treat joints from different individuals separately, we followed~\cite{duan2023skeletr,do2024skateformer} and reshape $\mathbf{X}$ to $T, V \cdot M, C$.
%To simplify the notation, we use $V$ in place of $V \cdot M$, thus our model input is $\mathbf{X} \in \mathbb{R}^{T \times V \times C}$.
Three linear layers project the low-dimensional skeleton data onto a higher-dimensional embedding space.
A learnable time-space token is added to the embedding, then input to $L$ Time-Space Mamba Blocks (\tsmb), each containing a Part-Group Mamba Block (\pgmb) modeling the skeletal-temporal relations and a feed-forward network (\ffn) for feature refinement.
To retain temporal dynamics information while reducing the computational complexity, after two \tsmb blocks, a \tdown layer consisting of a \convoned with a stride of 2 followed by \bn is applied.
The features obtained after the $L$ {\tsmb}s undergo skeletal-temporal average pooling followed by a \linear layer producing $\yhat \in \mathbb{R}^{\mathcal{C}}$, with $\mathcal{C}$ denoting the number of classes.
%$, trained via backpropagation against the true label \( y \), where \( N_c \) is the number of classes.
\subsection{Time-Space Mamba Block (\tsmb)}
To design our novel \tsmb, we follow a similar structure to traditional transformer blocks~\cite{vaswani2017attention}.
%It consists of a \pgmb, acting as a self-attention layer, followed by an \ffn, both with layer pre-normalization and skip connections.
The first part of the block models the spatial-temporal dynamics via part-grouped interactions
\begin{equation}
\mathbf{X} = \mathbf{X} + \pgmb(\ln(\mathbf{X}))
\end{equation}
with the \pgmb block computing
\begin{align}
[\mathbf{X}_{s}, \mathbf{X}_{m}, \mathbf{X}_{t}] &= \mysplit(\linear(\mathbf{X})) \\
\mathbf{X}_{s} &= \gconv(\mathbf{X}_{s}) \\
\mathbf{X}_{m} &= \pgml(\mathbf{X}_{m}) \\
\mathbf{X}_{t} &= \tconv(\mathbf{X}_{t}) \\
\mathbf{X} &= \linear(\myconcat(\mathbf{X}_{s}, \mathbf{X}_{m}, \mathbf{X}_{t}))
\end{align}
where \ln is the Layer Normalization operator, and \mysplit and \myconcat are the channel splitting and concatenation functions.
For input $\mathbf{X}$, we split it into $\mathbf{X}_{s}$, $\mathbf{X}_{m}$, and $\mathbf{X}_{t}$ with $C/4$, $C/2$, and $C/4$ channels, respectively.
%Each resulting tensor is processed, in parallel, by a different operator: $\mathbf{x}_{s}$ is processed by a \gconv layer modeling the spatial dynamics; $\mathbf{x}_{m}$ is analyzed by our novel \pgml capturing spatio-temporal relationships of different skeletal parts; $\mathbf{x}_{t}$ is exploited to capture temporal relationships by a \tconv layer.
%For input $\mathbf{x}$, we split \mathbf{x} into \(\mathbf{x}_{}\) with \(C/4\) channels, \(\mathbf{x}_{t}\) with \(C/4\) channels, and \(\mathbf{x}_{m}\) with \(C/2\) channels.
Following a similar spirit to multi-head attention mechanisms, we have $H/4$ parallel \pgml, \gconv, and \tconv operators, with $H$ denoting the number of heads.
The novel \pgml layer models spatial-temporal relationships of different body parts in $\mathbf{X}_{m}$.
Each \gconv is a one-layer GCN with a learnable $(H/4, V, V)$ matrix --instead of a predefined adjacency matrix-- that captures diverse joint spatial connectivity patterns in $\mathbf{X}_{s}$.
To model temporal dynamics of patterns in $\mathbf{X}_{t}$, every \tconv performs a $H/4$-grouped \convoned filtering operation, with kernel size $k_t$.
The second part of the block refines the captured skeletal spatial-temporal dynamics by computing
\begin{equation}
\mathbf{X} = \mathbf{X} + \underbrace{\linear(\gelu(\linear}_{\ffn}(\ln(\mathbf{X}))))
\end{equation}
which corresponds to the output of a \tsmb.
\subsection{Part-Grouped Mamba (\pgml)}
This layer is designed following three key insights:
(i) spatial (\gconv) and temporal (\tconv) operators model short-range temporal dynamics of the whole body;
(ii) different diseases affect specific body parts which exhibit distinct yet interrelated long-range temporal dynamics.
Following such intuitions, our novel \pgml introduces channel-wise driven scanning and part-based decomposition strategies in State Space Models~\cite{gu2022s4} to efficiently capture long-range local and global motion patterns.
\subsubsection{State Space Models (SSMs)}~are designed to map a 1D input sequence $x(t) \in \mathbb{R}$ to an output $y(t) \in \mathbb{R}$ using a hidden state $\mathbf{h}(t) \in \mathbb{R}^N$. Formally, this mapping is governed by the following ordinary differential equations (ODEs):
\begin{equation}
\label{eq:continuous_ssm}
\begin{aligned}
\mathbf{h}'(t) &= \mathbf{A}\mathbf{h}(t) + \mathbf{B}x(t) \\
y(t) &= \mathbf{C}\mathbf{h}(t)
\end{aligned}
\end{equation}
where \(\mathbf{A} \in \mathbb{R}^{N \times N}\) is the system’s evolution matrix, and \(\mathbf{B} \in \mathbb{R}^{N \times 1}\), \(\mathbf{C} \in \mathbb{R}^{1 \times N}\) are projection matrices.
Modern SSMs~\cite{gu2022s4} discretize~(\ref{eq:continuous_ssm}) using the zero-order hold (ZOH) method
\begin{equation}
\begin{aligned}
\mathbf{\overline{A}} &= \exp(\Delta \mathbf{A}) \\
\mathbf{\overline{B}} &= (\Delta \mathbf{A})^{-1} (\exp(\Delta \mathbf{A}) - \mathbf{I}) \cdot \Delta \mathbf{B}
\end{aligned}
\end{equation}
with timescale parameter $\Delta$ --which can be seen as the resolution of the continuous input $x(t)$-- leading to the discrete state-space equations
\begin{equation}
\begin{aligned}
\mathbf{h}_t &= \mathbf{\overline{A}}\mathbf{h}_{t-1} + \mathbf{\overline{B}}x_t \\
y_t &= \mathbf{C}\mathbf{h}_t
\end{aligned}
\end{equation}
that can be efficiently computed by the convolution
\begin{equation}
\begin{aligned}
\overline{\mathbf{K}} &= (\mathbf{C}\mathbf{\overline{B}}, \mathbf{C}\mathbf{\overline{A}}\mathbf{\overline{B}}, \cdots, \mathbf{C}\mathbf{\overline{A}}^{L-1}\mathbf{\overline{B}}) \\
\mathbf{y} &= \mathbf{x} * \mathbf{\overline{K}}
\end{aligned}
\end{equation}
where $L$ denotes the length of the input sequence $\mathbf{x}$ and $\mathbf{\overline{K}}$ is the SSM convolutional kernel.
Unlike traditional linear time-invariant SSMs, Mamba~\cite{guo2023ssmmamba} introduces a Selective Scan Mechanism (S6) such that parameters $\mathbf{B}$, $\mathbf{C}$, and $\Delta$ are directly derived from the input data, hence allowing input-dependent interactions along the sequence.
%thus allowing for adaptive weight modulation and contextual sensitivity.
%L$ and $D$ denote the length of the sequence and the feature dimensionality, respectively.
\subsubsection{Channel-Wise Spatio-Temporal SSM (C-2D-SSM):}
\begin{figure}[t]
\centering
\includegraphics[width=\linewidth]{figures/c2dssm.png}
\caption{Channel-Wise Spatio-Temporal SSM (C-2D-SSM).}
\label{fig:c2dssm}
\end{figure}
The Mamba architecture has been extended from 1D to 2D bidirectional modeling (\eg,~\cite{ma2024umamba,archit2024vimunet,xu2024mambasurvey}) showing promise in image-related tasks but exhibiting instability when scaled to large parameter spaces~\cite{patro2024simba}.
This is due to the Mamba block's extensive input and output projections, whose computational and parametric complexities scale linearly with the input channel dimensionality.
To effectively mitigate these issues, we leverage the insight that different channel groups in skeletal data may represent distinct yet complementary aspects of movement.
As shown in~\figurename~\ref{fig:c2dssm}, we decompose the input $\mathbf{X}$ along the channel dimension to get equally sized tensors
\begin{equation}
\left[ \mathbf{X}_{t \rightarrow s}, \mathbf{X}_{s \rightarrow t}, \mathbf{X}_{t \leftarrow s}, \mathbf{X}_{s \leftarrow t} \right] = \mysplit(\mathbf{X})
\end{equation}
that are independently processed by direction-specific 2D-SSMs (\ie, spatial-temporal SSMs), then concatenated back to produce the final output
\begin{equation}
\mathbf{Y} = \myconcat
\left(
\begin{array}{cc}
\ssm_{t \rightarrow s}(\mathbf{X}_{t \rightarrow s}), \\
\ssm_{s \rightarrow t}(\mathbf{X}_{s \rightarrow t}), \\
\ssm_{t \leftarrow s}(\mathbf{X}_{t \leftarrow s}), \\
\ssm_{s \leftarrow t}(\mathbf{X}_{s \leftarrow t}) \\
\end{array}
\right) = \mycssm(\mathbf{X})
\end{equation}
preserving the input tensor's dimensionality.
$t$ and $s$ denote temporal and spatial dimensions, respectively, and the arrows indicate the direction of spatial-temporal scanning.
%This novel parallel scanning approach captures complementary movement features across channel groups, enabling a more detailed global motion representation while maintaining computational efficiency.
This novel parallel approach captures complementary movement features across channel groups, while significantly reducing the computational complexity by operating on $C/4$ channel inputs.
The direction-specific spatial-temporal scanning mechanism effectively captures diverse contextual information, enhancing the model's capacity to learn both local and global dependencies.
\subsubsection{Part-Grouped Modeling:}~
%While existing models have extended the Mamba architecture from 1D to 2D through bidirectional modeling (\eg,~\cite{ma2024umamba,archit2024vimunet,xu2024mambasurvey}), they have not addressed the challenges of 3D spatial-temporal sequences in skeleton action recognition.
Joint locations change over time depending on the disease type.
Different deficits are caused by the involvement of multiple systems and structures (\eg, cerebellar, pyramidal, extrapyramidal).
For instance, hereditary paraplegia mostly involves lower body part joints~\cite{Serrao2016}, while Parkinson's and Cerbella Ataxia involve the whole body~\cite{10.1093/brain/122.7.1349}.
%Thus, it is important to capture the local and global skeletal-temporal dynamics to accurately model the intricate spatial relationships among 2D keypoints.
Motivated by these intuitions, we introduce both part-based and global SSMs to effectively capture fine-grained local motion details at individual body parts' level, while maintaining the global understanding of full-body motion.
%\subsubsection{Part-based SSMs:~}
% We decompose the input into multiple partitions corresponding to key body parts (arms, legs, torso) and their strategic combinations (arms-legs, arms-torso, torso-legs).
% For each skeleton keypoints partition $p \in {1, \cdots, P}$, we apply:
% \begin{equation}
% \mathbf{X}_p = \ssm_p(\mathbf{b}_p + \mathbf{X}[:, \mathcal{I}_p])
% \end{equation}
% where $\mathcal{I}_p$ is the index set for partition $p$, $\mathbf{b}_p \in \mathbb{R}^{T \times |\mathcal{I}_p| \times C}$ is a leanable partition token, and $\ssm_p$ represents a Mamba model tailored for that partition.
% This mechanism allows each SSM to learn a unique representation of the motion dynamics of each body part.
%Following biomechanical principles and the hierarchical nature of human motion,
We decompose the body keypoints into multiple partitions corresponding to key body parts (arms, legs, torso) and their relevant combinations (arms-legs, arms-torso, torso-legs).
By focusing on these specific partitions, we enable our model to capture both localized movements and inter-segment temporal dynamics crucial for disease recognition.
For each partition $p \in \{1, \cdots, P\}$, we apply:
\begin{equation}
\mathbf{X}_p = \mycssm_p(\mathbf{b}_p + \mathbf{X}[:, \mathcal{I}_p])
\end{equation}
where $\mathcal{I}_p$ is the index set, $\mathbf{b}_p \in \mathbb{R}^{T \times |\mathcal{I}_p| \times C}$ is a learnable partition token, and $\mycssm_p$ is our novel C-2D-SSM for partition $p$.
The learnable partition token $\mathbf{b}_p$ enables each SSM to learn a specialized representation of the motion dynamics specific to each body part or combination, enhancing the model's ability to differentiate between fine-grained movements.
By processing these partitions separately, our model can efficiently capture part-specific temporal patterns, such as the rhythmic alternation of arm and leg movements during walking, or the stabilizing role of the torso in maintaining balance.
%The strategic combinations (arms-legs, arms-torso, torso-legs) further allow the model to learn higher-order motion correlations, crucial for understanding complex actions like sports movements or dance performances.
%This part-based approach not only enhances the model's motion understanding capabilities but also introduces a strong anatomically-informed inductive bias, potentially improving generalization to unseen motion sequences.
%\subsubsection{Global SSM:~}
To ensure that full-body motion patterns are not missed when focusing solely on individual parts, we capture holistic motion characteristics as $\mathbf{X}_g = \mycssm(\mathbf{X})$.
\subsubsection{Attentive SSM:~}
The outputs from these specialized SSMs are then integrated through a learnable weighted sum:
\begin{equation}
\mathbf{X}_{\text{SSM}} = \sum_p \beta_p \mathbf{X}_p + \beta_g \mathbf{X}_g
\end{equation}
where $\beta_p$ and $\beta_g$ are learnable parameters.
This integration strategy allows our model to dynamically adjust the importance of part-specific and global motion information based on the input data.
% The part-based modeling approach offers several compelling advantages:
% Enhanced Motion Detail: By focusing on specific body parts and their interactions, our model captures fine-grained motion details that might be lost in a purely global representation. This is particularly crucial for tasks requiring high precision, such as action recognition or motion synthesis.
% Spatial Attention Mechanism: The part-based decomposition acts as an implicit form of spatial attention, enabling the model to dynamically weight the importance of different body regions. This is especially beneficial for actions where certain body parts are more informative than others.
% Structured Inductive Bias: By incorporating knowledge about human body structure into the model architecture, we introduce a strong inductive bias. This structured approach enhances the model's ability to generalize across different types of motions and subjects.
% Computational Efficiency: Processing smaller, part-based representations in parallel allows for more efficient computation compared to processing the entire body at once, without sacrificing the ability to capture global patterns.
To further refine our feature representations and enhance the model's adaptability, we incorporate a channel attention mechanism after the SSM processing.
We first compute
\begin{equation}
\mathbf{w} = \sigma(\linear(\gelu(\linear(\pool(\mathbf{X}_{\text{SSM}})))))
\end{equation}
where $\pool$ is the spatial-temporal average pooling operator and $\sigma$ is the sigmoid function.
Then, we obtain the output of our novel \pgml as
\begin{equation}
\pgml(\mathbf{X}) = (\mathbf{X}_{\text{SSM}} + \mathbf{X}) \odot \mathbf{w}
\end{equation}
where $\alpha$ is a learnable parameter and $\odot$ denotes the Hadamard product.
The residual connection ($+ \mathbf{X}$) ensures efficient gradient flow during training, while the channel attention ($\odot \mathbf{w}$) allows our model to adaptively recalibrate channel-wise feature responses~\cite{hu2018squeeze}, focusing on the most informative features for each input sequence.
%This comprehensive architecture, combining part-based selective state-space modeling with channel attention, represents a significant advancement in motion analysis. By leveraging the strengths of both local and global processing, our model achieves a nuanced understanding of complex motion patterns. The result is a powerful yet computationally feasible approach that pushes the boundaries of performance in tasks such as human motion analysis, action recognition, and multi-object tracking in video data.
\section{Experiments}
\subsection{Datasets}
\subsubsection{Medical Diagnosis:}~Our objective is to provide a method for automated diagnosis of motion-related disorders.
To assess the capabilities of our method in such a context, we validate it on one newly collected dataset and a publicly available benchmark.
Both datasets present a challenging scenario for automated diagnosis from skeleton action recognition, requiring models to differentiate between multiple neurological conditions and healthy controls based on subtle motion characteristics.
\\
\textbf{Neurological Disorders (ND).}~
We collected a new dataset focused on automated diagnosis of neurological disorders.
Our dataset comprises 396 video sequences from 40 subjects across four distinct classes: primary degenerative cerebellar ataxia (11 patients, 112 sequences), hereditary spastic paraparesis (12 patients, 105 sequences), idiopathic Parkinson's disease (7 patients, 80 sequences), and healthy controls (10 subjects, 99 sequences).
Data collection was conducted under carefully controlled conditions to ensure consistency and reliability. Each sequence captures a patient walking in a standardized environment, with multiple recordings per subject in both directions - towards and away from the camera.
All sequences were recorded using an HD camera operating at 30 fps.
The average sequence length is 140.64 frames, with a range from 69 to 465 frames, providing sufficient temporal context for analyzing motion characteristics associated with each condition.
\\
\textbf{KOA-PD-NM}~\cite{kour2024koanmpddataset} contains marker-based sequences of subjects with Knee Osteoarthritis (KOA, 50 patients), Parkinson's Disease (PD, 20 patients), and Normal (NM, 30 patients) motion patterns.
This dataset was captured in a controlled indoor environment (subjects walked on a green mat in the sagittal plane) using a 50fps HD camera.
The dataset includes varying severity levels (mild, moderate, severe) for KOA and PD subjects, graded by medical specialists.
We report the results computed considering and excluding the severity levels (\ie, yielding a 7-class and a 3-class dataset, respectively).
\input{tables/sota_comparison_medical}
\subsubsection{Generic Actions:}~
While our approach is primarily designed for automated diagnosis of motion-related neurological disorders, we hypothesize its potential in generic action recognition tasks. To validate this hypothesis and provide a comprehensive evaluation, we conduct experiments on established public benchmarks for skeleton-based action recognition.
\\
\textbf{NTU RGB+D}~\cite{shahroudy2016ntu60} comprises 60 action classes, encompassing a wide range of daily activities.
It contains 56,880 video samples captured from 40 subjects across 155 camera viewpoints using Kinect v2, providing RGB, IR, depth, and 3D skeleton data. The dataset supports cross-subject (X-Sub60) and cross-view (X-View60) evaluation protocols.
11 classes involve two-person interactions, forming the NTU-Inter subset~\cite{duan2023skeletr,pang2022igformer,do2024skateformer}.
\\
\textbf{NTU RGB+D 120}~\cite{liu2020ntu120} extends the NTU RGB+D dataset to 120 action classes with 114,480 video samples from 106 subjects.
It introduces cross-subject (X-Sub120) and cross-setup (X-Set120) evaluation protocols.
The dataset includes 26 classes focused on human interaction, designated as NTU-Inter 120 \cite{duan2023skeletr,pang2022igformer,do2024skateformer}.
\\
\textbf{NW-UCLA}~\cite{wang2014nwucla} consists of 1,475 video samples spanning 10 action classes, performed by 10 subjects and captured from three distinct camera views. It provides RGB, IR, depth, and 3D skeleton data. Evaluation is conducted using a cross-view protocol~\cite{zhou2022hyperformer,do2024skateformer}, utilizing two views for training and one for testing.
\subsection{Implementation Details}
% All experiments were performed using the PyTorch framework [35], running on a single NVIDIA L40S GPU.
% Each model was trained with a total of 500 epochs. We employed a linear warm-up strategy for the learning rate, gradually
% increasing it from 10−7 to 10−3 during the first 25 epochs.
% Subsequently, a cosineannealing scheduler [31] was used to update the learning rate at each iteration for the remaining epochs. We employed the AdamW optimizer [32] with a betas of (0.9, 0.999), a weight decay of 5 × 10−4. Additionally, gradient clipping [70] was applied for the loss values with gradients exceeding 1.
% The batch size was set to 128, the random seed was fixed at 1, and we adopted the label-smoothed cross-entropy loss [33, 52] as our loss function with parameter α = 0.1.
% From various experiments, we found the empirical values as follows.
% For our experiments with the NTU RGB+D and NTU RGB+D 120 datasets, we used the following configuration: V = 48 (excluding the center of the body, resulting in 24 joints for each of the two individuals), T = 64, L = 4, K = 12, M = 8, N = 8, and C = 96. For the experiments with the NW-UCLA dataset, V = 20, T = 64, L = 4, K = 5, M = 8, N = 8, and C = 96.
We trained our model on an NVIDIA L40S GPU using the PyTorch framework for 500 epochs on 128-sample batches, using the AdamW optimizer with weight decay equal to $5e-4$.
We adopted a linear learning rate warm-up from 1e-7 to 1e-3 over the first 25 epochs, followed by a cosine annealing scheduler.
We applied clipping for gradients having $\ell_2$-norm exceeding 1.
The label-smoothed cross-entropy loss with $\alpha= 0.1$ was used for optimization.
For the ND and KOA-PD-NM datasets, we used~\cite{sun2019hrnet} to obtain the skeleton joints.
For others, we used the provided keypoints.
\subsection{State-of-the-art Comparison}
We present a comprehensive performance comparison of our method against recent state-of-the-art skeleton-based action recognition methods.
Following~\cite{zhou2023, do2024skateformer,zhou2022hyperformer}, we considered three different modalities:
(i) only joints ($\modalityj$)
(ii) joints and bones ($\modalityjb$);
and (iii) joints and bones with motions ($\modalityjbm$).
We trained a model for each modality and ensembled their outputs.
%\input{tables/sota_comparison_koa}
%\input{tables/sota_comparison_uniroma1}
\subsubsection{Medical Diagnosis:}~
Table~\ref{tab:sota_medical} provides a comparative evaluation of different models on three medical diagnosis datasets
Our proposed model achieves the highest accuracy across all datasets and ensemble metrics.
For the ND dataset, we reach 99.35\% on $\modalityj$ and $\modalityjb$, further improving to 99.64\% on $\modalityjbm$, demonstrating robust performance in diagnosis of neurological disorders.
In the KOA-PD-NM dataset, we significantly improve other methods reading a 98.62\% accuracy using the $\modalityjbm$.
This indicates a strong capability in classifying grouped severity levels.
On the more challenging KOA-PD-NM-Severity dataset, our method better handles fine-grained severity distinctions outperforming all existing models.
Our novel state-space model architecture achieves substantial performance gains across datasets, highlighting its effectiveness for nuanced medical diagnosis tasks that require precise classification of disease severity.
This is likely due to the model's ability to dynamically capture long-range local and global joint interactions.
\subsubsection{Generic Actions:}
\input{tables/sota_comparison_generic}
\input{tables/sota_comparison_generic_inter}
Results in Table~\ref{tab:sota_generic} show that our noble method achieves leading performance across diverse generic action recognition benchmarks.
We consistently outperformed prior methods, demonstrating the effectiveness of our modeling approach.
On the NTU RGB+D X-Sub60 dataset, we recorded top scores of 91.8\%, 92.8\%, and 93.4\% across the three considered ensemble strategies, showing the capabilities of our approach in capturing spatial-temporal dependencies.
For the more complex NTU RGB+D 120 dataset, our method demonstrates excellent performance by improving over prior GCN and Transformer-based methods.
Similarly, on the NW-UCLA dataset, we obtain the highest accuracy at $97.6\%$ on the leaderboard.
%All such results substantiate the capabilities of our method to handle expanded and diverse action classes.
%, likely due to its efficient feature partitioning that allows the model to learn distinct spatiotemporal representations.
Similar comments can be made for the results obtained comparing our method with state-of-the-art human interaction recognition methods on the NTU-inter and NTU-Inter 120 datasets.
Table~\ref{tab:sota_generic_inter} shows that we consistently achieve superior performance than existing methods, demonstrating the robustness and effectiveness of our approach in handling complex human interactions.
\subsubsection{Complexity Analysis:}
\input{tables/sota_complexity_analysis}
Table~\ref{tab:sota_complexity_analysis} provides a comparative computational complexity analysis with existing architectures.
Our approach achieves the highest accuracy, recording an average joint modality accuracy of 94.3\% and 88.1\% on the NTU RGB+D and NTU RGB+D 120 datasets respectively --surpassing all existing methods.
While SkelMamba exhibits a slightly higher parameter count (6.84M) and FLOPs (9.7G) than some lighter GCN models, its inference time of 7.06 ms shows it is remarkably efficient. This trade-off positions our method as the most accurate and fastest model.
\begin{table*}[t]
\centering
\caption{Comparison with existing methods on the three considered medical diagnosis benchmark datasets: ND, KOA-PD-NM (3 classes with grouped severity levels), and KOA-PD-NM-Severity (7 classes with different diseases' severity levels).
Results are shown considering the number of input frames and ensemble strategies.}
\label{tab:sota_medical}
\footnotesize
\begin{tabularx}{\linewidth}{Y l Y Y Y Y Y Y Y Y Y}
\toprule
& & \multicolumn{3}{c}{\textbf{ND}} & \multicolumn{3}{c}{\textbf{KOA-PD-NM}} & \multicolumn{3}{c}{\textbf{KOA-PD-NM-Severity}} \\
\cmidrule(lr){3-5} \cmidrule(lr){6-8} \cmidrule(lr){9-11}
Type & Model & $\modalityj$ & $\modalityjb$ & $\modalityjbm$ & $\modalityj$ & $\modalityjb$ & $\modalityjbm$ & $\modalityj$ & $\modalityjb$ & $\modalityjbm$ \\
\midrule
CNN & PoseC3D~\cite{duan22posec3d} & 98.95 & 99.21 & 99.35 & 84.79 & 89.40 & 89.86 & 95.79 & 98.95 & 98.95 \\
\hline
\multirow{2}{*}{GCN} & 2S-AAGCN~\cite{shi2019agcn} & 94.74 & 97.89 & 98.95 & 91.22 & 92.39 & 93.18
& 89.47 & 93.68 & 93.68 \\
& STGCN++~\cite{yan2018stgcn} & 93.68 & 94.74 & 95.79 & 93.54 & 94.31 & 95.32 & 95.32 & 95.79 & 96.84 \\
\hline
Transformer & Hyperformer~\cite{zhou2022hyperformer} & 99.21 & 99.35 & 99.21 & 96.82 & 96.94 & 98.14 & 95.79 & 96.84 & 98.95 \\
\hline
\rowcolor{gray!25}
\cellcolor{white}
SSM & \textbf{SkelMamba (Ours)} & \textbf{99.35} & \textbf{99.35} & \textbf{99.64} & \textbf{97.45} & \textbf{98.23} & \textbf{98.62} & \textbf{96.84 }& \textbf{98.95} & \textbf{99.21} \\
\bottomrule
\end{tabularx}
\end{table*}
\begin{table*}[ht]
\centering
\caption{Comparison with existing methods on the three considered generic actions benchmark datasets: NTU RGB+D, NTU RGB+D 120, and NW-UCLA.
Results are shown considering the number of input frames and ensemble strategies.}
\label{tab:sota_generic}
\footnotesize
\begin{tabularx}{\linewidth}{Y l Y YYY YYY YYY YYY Y}
\toprule
\multirow{3}{*}{Types} & \multirow{3}{*}{Methods} & \multirow{3}{*}{Frames} & \multicolumn{6}{c}{\textbf{NTU RGB+D}} & \multicolumn{6}{c}{\textbf{NTU RGB+D 120}} & \textbf{NW} \\
& & & \multicolumn{3}{c}{X-Sub60} & \multicolumn{3}{c}{X-View60} & \multicolumn{3}{c}{X-Sub120} & \multicolumn{3}{c}{X-Set120} & \textbf{UCLA}\\
\cmidrule(lr){4-6} \cmidrule(lr){7-9} \cmidrule(lr){10-12} \cmidrule(lr){13-15} \cmidrule(lr){16-16}
& & & $\modalityj$ & $\modalityjb$ & $\modalityjbm$ & $\modalityj$ & $\modalityjb$ & $\modalityjbm$ & $\modalityj$ & $\modalityjb$ & $\modalityjbm$ & $\modalityj$ & $\modalityjb$ & $\modalityjbm$ & \\
\midrule
\multirow{1}{*}{RNN} & AGC-LSTM~\cite{si2019agclstm} & 100 & 87.5 & 89.2 & - & 93.5 & 95.0 & - & - & - & - & - & - & - & 93.3 \\
\hline
\multirow{2}{*}{CNN} & TA-CNN~\cite{xu2022tacnn} & 64 & 88.8 & - & 90.4 & 93.6 & - & 94.8 & 82.4 & - & 85.4 & 84.0 & - & 86.8 & 96.1 \\
& Ske2Grid~\cite{cai2023ske2grid} & 100 & 88.3 & - & - & 95.7 & - & - & 82.7 & - & - & 85.1 & - & - & - \\
\hline
\multirow{10}{*}{GCN} & SGN~\cite{zhang2020sgn} & 20 & - & 89.0 & - & - & 94.5 & - & 79.2 & - & - & 81.5 & - & - & - \\
& CTR-GCN~\cite{chen2021ctrgcn} & 64 & 89.9 & - & 92.4 & - & - & 96.8 & 84.9 & 88.7 & 88.9 & - & 90.1 & 90.6 & 96.5 \\
& ST-GCN++~\cite{yan2018stgcn} & 100 & 89.3 & 91.4 & 92.1 & 95.6 & 96.7 & 97.0 & 83.2 & 87.0 & 87.5 & 85.6 & 87.5 & 89.8 & - \\
& InfoGCN~\cite{chi2022infogcn} & 64 & - & - & 92.7 & - & - & 96.9 & 85.1 & 88.5 & 89.4 & 86.3 & 89.7 & 90.7 & 96.6 \\
& FR-Head~\cite{zhou2023} & 64 & 90.3 & 92.3 & 92.8 & 95.3 & 96.4 & 96.8 & 85.5 & - & 89.5 & 87.3 & - & 90.9 & 96.8 \\
& Koopman~\cite{wang2023koopman} & 64 & 90.2 & - & 92.9 & 95.2 & - & 96.8 & 85.7 & - & 90.0 & 87.4 & - & 91.3 & 97.0 \\
& LST~\cite{xiang2023lst} & 64 & 90.2 & - & 92.9 & 95.6 & - & 97.0 & 85.5 & - & 89.9 & 87.0 & - & 91.1 & 97.2 \\
& HD-GCN~\cite{lee2023hdgcn} & 64 & 90.6 & 92.4 & 93.0 & 95.7 & 96.6 & 97.0 & 85.7 & 89.1 & 89.8 & 87.3 & 90.6 & 91.2 & 96.9 \\
& STC-Net~\cite{lee2023stcnet} & 64 & - & 92.5 & 93.0 & - & 96.7 & 97.1 & - & 89.3 & 89.9 & - & 90.7 & 91.3 & 97.2 \\
& BlockGCN~\cite{zhou2024blockgcn} & 64 & 90.9 & - & 93.1 & 95.4 & - & 97.0 & 86.9 & - & \textbf{90.3} & 88.2 & - & 91.5 & 96.9 \\
\hline
\multirow{4}{*}{Transf.} & DSTA-Net~\cite{shi2020dstanet} & 128 & - & - & 91.5 & - & - & 96.4 & - & - & 86.6 & - & - & 89.0 & - \\
& STST~\cite{zhang2021stst} & 128 & - & - & 91.9 & - & - & 96.8 & - & - & - & - & - & - & - \\
& FG-STFormer~\cite{gao2022fgstformer} & 128 & - & - & 92.6 & - & - & 96.7 & - & - & 89.0 & - & - & 90.6 & 97.0 \\
& Hyperformer~\cite{zhou2022hyperformer} & 64 & 90.7 & - & 92.9 & 95.1 & - & 96.5 & 86.6 & - & 88.0 & 88.0 & - & 91.3 & 96.7 \\
%& FreqMixFormer~\cite{wu2024freqaware} & 64 & 91.5 & -& \textbf{93.4} & 96.0 &- & 97.3 & \textbf{87.9} & - & \textbf{90.2} & \textbf{89.1} & - & \textbf{91.5} & - \\
%& SkateFormer~\cite{do2024skateformer} & 64 & 92.6 & 93.0 & 93.5 & 97.4 & 97.0 & 97.4 & 87.7 & 89.4 & 89.8 & 89.3 & 91.0 & 91.4 & 98.3 \\
\hline
\multirow{2}{*}{SSM} & Simba~\cite{patro2024simba} & 64 & 89.0 & 90.5 & - & 94.4 & 95.2 & - & 79.7 & - & - & 86.3 & - & - & 96.3 \\
\rowcolor{gray!25}
\cellcolor{white}
& \textbf{SkelMamba (Ours)} & 64 & \textbf{91.8} & \textbf{92.8} & \textbf{93.4} & \textbf{96.8} & \textbf{97.1} & \textbf{97.4} & \textbf{87.1} & \textbf{89.4} & 89.9 &\textbf{ 89.1} & \textbf{90.7} & \textbf{91.5} & \textbf{97.6} \\
\bottomrule
\end{tabularx}%
\end{table*}
\begin{table*}[ht]
\centering
\caption{Comparison with existing human interaction recognition methods on the NTU-Inter
and NTU-Inter 120 datasets.}
\label{tab:sota_generic_inter}
\footnotesize
\begin{tabularx}{\linewidth}{Y l Y Y Y Y Y Y Y}
\toprule
\multirow{2}{*}{Types} & \multirow{2}{*}{Methods} & \multicolumn{2}{c}{NTU-Inter ($\modalityj$, \%)} & \multicolumn{2}{c}{NTU-Inter 120 ($\modalityj$, \%)} & \multirow{2}{*}{Params [M]} & \multirow{2}{*}{FLOPs [G]} & \multirow{2}{*}{Time [ms]} \\
\cline{3-4} \cline{5-6}
& & X-Sub60 & X-View60 & X-Sub120 & X-Set120 & & & \\
\hline
\multirow{3}{*}{Transformer} & IGFormer~\cite{pang2022igformer} & 93.6 & 96.5 & 85.4 & 86.5 & - & - & - \\
& SkeleTR~\cite{duan2023skeletr} & 94.9 & 97.7 & 87.8 & 88.3 & 3.82 & 7.30 & - \\
& ISTA-Net~\cite{wen2023istanet} & - & - & 90.6 & 91.7 & 6.22 & 68.18 & 21.71 \\
\hline
\rowcolor{gray!25}
\cellcolor{white}
SSM & \textbf{SkelMamba (Ours)} & \textbf{96.5} & \textbf{98.9} & \textbf{92.0} & \textbf{92.8} & 6.84 & 9.7 & 7.06 \\
\bottomrule
\end{tabularx}%
\end{table*}
\begin{table*}[t]
\centering
\caption{Computational analysis: comparison of parameters, FLOPs, inference time, and average top-1 accuracy for joint modality ($\uparrow$ higher is better, $\downarrow$ lower is better).}
\label{tab:sota_complexity_analysis}
\footnotesize
\begin{tabularx}{\linewidth}{Y l Y Y Y c c}
\toprule
Types & Methods & Params [M]$\downarrow$& FLOPs [G] $\downarrow$& Time [ms]$\downarrow$ & NTU RGB+D [\%]$\uparrow$& NTU RGB+D 120 [\%]$\uparrow$\\
\midrule
\multirow{5}{*}{GCN} & InfoGCN~\cite{chi2022infogcn} & 1.56 & 3.34 & 12.97 & - & 85.7 \\
& FR-Head~\cite{zhou2023} & 1.45 & 3.60 & 18.49 & 92.8 & 86.4 \\
& Koopman~\cite{wang2023koopman} & 5.38 & 8.76 & 17.86 & 92.7 & 86.6 \\
& LST~\cite{xiang2023lst} & 2.10 & 3.60 & 18.85 & 92.9 & 86.3 \\
& HD-GCN~\cite{lee2023hdgcn} & 1.66 & 3.44 & 72.81 & 93.2 & 86.5 \\
\hline
\multirow{2}{*}{Transformer} & DSTA-Net~\cite{shi2020dstanet} & 3.45 & 16.18 & 13.80 & - & - \\
& Hyperformer~\cite{zhou2022hyperformer} & 2.71 & 9.64 & 18.07 & 92.9 & 87.3 \\
\hline
\rowcolor{gray!25}
\cellcolor{white}
SSM & \textbf{SkelMamba (Ours)} & 6.84 & 9.7 & 7.06 & \textbf{94.3} & \textbf{88.1} \\
\bottomrule
\end{tabularx}%
\end{table*}
\subsection{Ablation Study}
\subsubsection{Part-Grouped Mamba Block Components:}~
\input{tables/ablation_components}
Table~\ref{tab:ablation_components} presents an ablation study on the novel part-grouped mamba block, investigating the contributions of its spatial, temporal, and spatio-temporal components.
The ablation of either the spatial or temporal component leads to a significant drop in accuracy, highlighting the importance of both modalities for the block's performance.
The PGM module, which models interactions of different parts of the body, consistently improves accuracy across all configurations.
This suggests that the PGM effectively captures the spatial relationships between body parts.
%Overall, the results demonstrate the effectiveness of the novel part-grouped Mamba block in capturing rich spatio-temporal information, leading to improved performance on the NTU RGB+D dataset.
\subsubsection{Body partitioning and SSM scanning:}~
\input{tables/ablation_pgm}
Table~\ref{tab:ablation_pgm}~ presents the ablation study of the Part Grouped Mamba ($\pgml$) layer.
Transitioning from 1D SSM to C-2D-SSM notably improves performance, highlighting the benefit of spatial-temporal dependency modeling.
Incorporating attentive SSM enhances performance by capturing long-range dependencies, while channel attention refines feature representation, yielding additional gains.
Altogether, the fully integrated PGM layer achieves the highest accuracy, validating its design for robust part-based skeleton action recognition.
\begin{table*}[t]
\centering
\caption{Ablation analysis of the spatial, temporal, and spatio-temporal components of the novel part-grouped mamba block. Results show the top-1 accuracy for the $\modalityj$ modality ($\uparrow$ higher is better, $\downarrow$ lower is better).}
\label{tab:ablation_components}
\footnotesize
\begin{tabularx}{\linewidth}{Y Y Y l Y Y Y c}
\toprule
\multicolumn{3}{c}{Components} & \multicolumn{2}{c}{NTU RGB+D [\%] $\uparrow$} & \multirow{2}{*}{Params [M]$\downarrow$} & \multirow{2}{*}{FLOPs [G]$\downarrow$} & \multirow{2}{*}{Time [ms]$\downarrow$} \\ \cmidrule{1-3} \cmidrule{4-5}
\gconv & \tconv & \pgml & X-Sub60 & X-View60 & & & \\
\midrule
\tick & & & 89.9 & 94.6 & 4.3 & 8.2 & 2.65 \\
& \tick & & 87.2 & 93.5 & 4.1 & 8.1 & 1.80 \\
& & \tick & 90.9 & 95.1 & 6.6 & 9.5 & 6.90 \\
\tick & \tick & & 90.6 & 94.8 & 4.3 & 8.2 & 2.99 \\
\tick & & \tick & 91.0 & 95.8 & 6.7 & 9.4 & 3.85 \\
& \tick & \tick & 91.1 & 95.4 & 6.6 & 9.4 & 5.76 \\
\rowcolor{gray!25} \tick & \tick & \tick & \textbf{91.8} & \textbf{96.8} & 6.8 & 9.7 & 7.06 \\
\bottomrule
\end{tabularx}%
\end{table*}
\begin{table}[t]
\centering
\caption{Contribution of the introduced components of the novel part grouped mamba (\pgml) layer. Results show the top-1 accuracy for the $\modalityj$ modality.}
\label{tab:ablation_pgm}
\footnotesize
\begin{tabularx}{\linewidth}{l Y Y}
\toprule
\multirow{2}{*}{\pgml} & \multicolumn{2}{c}{NTU RGB+D [\%]} \\ \cmidrule{2-3}
& X-Sub60 & X-View60 \\
\midrule
1D SSM Baseline & 88.6 & 94.1 \\
+ 1D SSM to C-2D-SSM & 90.2 (+1.6) & 95.3 (+1.2) \\
+ Partition Token ($\mathbf{b}_p$) & 90.5 (+0.3) & 95.6 (+0.3) \\
+ Attentive SSM ($\mathbf{X}_{\text{SSM}}$) & 91.6 (+1.1) & 96.4 (+0.8) \\
+ Channel attention ($\mathbf{w}$) & \textbf{91.8} (+0.7) & \textbf{96.8} (+0.4) \\
\bottomrule
\end{tabularx}%
\end{table}
\section{Conclusion}
We presented a novel skeleton-based action recognition framework that integrates anatomically-aware State Space Models (SSMs) for fine-grained analysis of spatio-temporal motion patterns. Our approach introduces a multi-stream architecture that partitions skeletal data into spatial, temporal, and spatio-temporal streams, enabling efficient and targeted analysis of complex human motions. Through anatomically-guided body part segmentation and a multi-directional scanning strategy, our method captures both local joint dynamics and global motion interactions crucial for applications requiring precision, such as automated medical diagnosis.
Extensive evaluations on standard benchmarks (NTU RGB+D, NTU RGB+D 120, NW-UCLA) demonstrate our model’s superiority by significantly improving over existing methods.
On a new medical dataset for gait analysis, our framework also exhibits strong potential for clinical diagnostics, accurately identifying subtle movement patterns indicative of neurological disorders.
However, our medical dataset remains limited in size, as expanding it requires complex data-gathering processes constrained by privacy concerns and regulatory restrictions. We are actively working to increase this dataset, but this endeavor remains resource-intensive.
\begin{thebibliography}{77}
\providecommand{\natexlab}[1]{#1}
\providecommand{\url}[1]{\texttt{#1}}
\expandafter\ifx\csname urlstyle\endcsname\relax
\providecommand{\doi}[1]{doi: #1}\else
\providecommand{\doi}{doi: \begingroup \urlstyle{rm}\Url}\fi
\bibitem[Adeli et~al.(2024)Adeli, Mehraban, Ballester, Zarghami, Sabo, Iaboni, and Taati]{adeli2024benchmarkingskeletonparkinson}
Vida Adeli, Soroush Mehraban, Irene Ballester, Yasamin Zarghami, Andrea Sabo, Andrea Iaboni, and Babak Taati.
\newblock Benchmarking skeleton-based motion encoder models for clinical applications: Estimating parkinson's disease severity in walking sequences, 2024.
\bibitem[Archit and Pape(2024)]{archit2024vimunet}
Anwai Archit and Constantin Pape.
\newblock Vim-unet: Vision mamba for biomedical segmentation.
\newblock In \emph{MIDL}, 2024.
\bibitem[Cai et~al.(2023)Cai, Kang, Yao, and Chen]{cai2023ske2grid}
Dongqi Cai, Yangyuxuan Kang, Anbang Yao, and Yurong Chen.
\newblock Ske2grid: Skeleton-to-grid representation learning for action recognition.
\newblock In \emph{International Conference on Machine Learning}, 2023.
\bibitem[Cai et~al.(2021)Cai, Jiang, Han, Jia, and Lu]{cai2021jologcn}
Jinmiao Cai, Nianjuan Jiang, Xiaoguang Han, Kui Jia, and Jiangbo Lu.
\newblock Jolo-gcn: Mining joint-centered light-weight information for skeleton-based action recognition.
\newblock In \emph{IEEE/CVF Winter Applications of Computer Vision}, 2021.
\bibitem[Chaudhuri and Bhattacharya(2024)]{chaudhuri2024simba}
Soumyabrata Chaudhuri and Saumik Bhattacharya.
\newblock Simba: Mamba augmented u-shiftgcn for skeletal action recognition in videos.
\newblock \emph{ArXiV Preprint}, 2024.
\bibitem[Chen et~al.(2024)Chen, Huang, Xu, Pei, Chen, Li, Wang, Li, Lu, and Wang]{chen2024videomamba}
Guo Chen, Yifei Huang, Jilan Xu, Baoqi Pei, Zhe Chen, Zhiqi Li, Jiahao Wang, Kunchang Li, Tong Lu, and Limin Wang.
\newblock Video mamba suite: State space model as a versatile alternative for video understanding, 2024.
\bibitem[Chen et~al.(2021)Chen, Zhang, Yuan, Li, Deng, and Hu]{chen2021ctrgcn}
Yuxin Chen, Ziqi Zhang, Chunfeng Yuan, Bing Li, Ying Deng, and Weiming Hu.
\newblock Channel-wise topology refinement graph convolution for skeleton-based action recognition.
\newblock In \emph{International Conference on Computer Vision}, 2021.
\bibitem[Cheng et~al.(2020)Cheng, Zhang, He, Chen, Cheng, and Lu]{cheng2020shiftgcn}
Ke Cheng, Yifan Zhang, Xiangyu He, Weihan Chen, Jian Cheng, and Hanqing Lu.
\newblock Skeleton-based action recognition with shift graph convolutional network.
\newblock In \emph{IEEE/CVF International Conference on Computer Vision and Pattern Recognition}, 2020.
\bibitem[Chi et~al.(2022)Chi, Ha, Chi, Lee, Huang, and Ramani]{chi2022infogcn}
Hyung-Gun Chi, Hoon Ha, Seunggeun Chi, Sang~Wan Lee, Qixing Huang, and Karthik Ramani.
\newblock Infogcn: Representation learning for human skeleton-based action recognition.
\newblock In \emph{IEEE/CVF International Conference on Computer Vision and Pattern Recognition}, 2022.
\bibitem[Deng and Gu(2024)]{deng2024cumamba}
Rui Deng and Tianpei Gu.
\newblock Cu-mamba: Selective state space models with channel learning for image restoration, 2024.
\bibitem[Do and Kim(2024)]{do2024skateformer}
Jeonghyeok Do and Munchurl Kim.
\newblock Skateformer: Skeletal-temporal transformer for human action recognition.
\newblock In \emph{European Conference on Computer Vision}, 2024.
\bibitem[Duan et~al.(2022{\natexlab{a}})Duan, Wang, Chen, and Lin]{duan2022pyskl}
Haodong Duan, Jiaqi Wang, Kai Chen, and Dahua Lin.
\newblock Pyskl: Towards good practices for skeleton action recognition.
\newblock \emph{arXiv preprint arXiv:2205.09443}, 2022{\natexlab{a}}.
\bibitem[Duan et~al.(2022{\natexlab{b}})Duan, Zhao, Chen, Lin, and Dai]{duan22posec3d}
Haodong Duan, Yue Zhao, Kai Chen, Dahua Lin, and Bo Dai.
\newblock Revisiting skeleton-based action recognition.
\newblock In \emph{IEEE/CVF Conference on Computer Vision and Pattern Recognition}, pages 2959--2968. IEEE, 2022{\natexlab{b}}.
\bibitem[Duan et~al.(2023)Duan, Xu, Shuai, Modolo, Tu, Tighe, and Bergamo]{duan2023skeletr}
Haodong Duan, Mingze Xu, Bing Shuai, Davide Modolo, Zhuowen Tu, Joseph Tighe, and Alessandro Bergamo.
\newblock Skeletr: Towards skeleton-based action recognition in the wild.
\newblock In \emph{IEEE/CVF International Conference on Computer Vision}, 2023.
\bibitem[Ebersbach et~al.(1999)Ebersbach, Sojer, Valldeoriola, Wissel, Müller, Tolosa, and Poewe]{10.1093/brain/122.7.1349}
G. Ebersbach, M. Sojer, F. Valldeoriola, J. Wissel, J. Müller, E. Tolosa, and W. Poewe.
\newblock {Comparative analysis of gait in Parkinson's disease, cerebellar ataxia and subcortical arteriosclerotic encephalopathy}.
\newblock \emph{Brain}, 122\penalty0 (7):\penalty0 1349--1355, 1999.
\bibitem[Fu et~al.(2022)Fu, Dao, Saab, Thomas, Rudra, and Ré]{fu2022hungry}
Daniel~Y. Fu, Tri Dao, Khaled~K. Saab, Armin~W. Thomas, Atri Rudra, and Christopher Ré.
\newblock Hungry hungry hippos: Towards language modeling with state space models.
\newblock \emph{ArXiV Preprint}, 2022.
\bibitem[Gu and Dao(2023)]{guo2023ssmmamba}
Albert Gu and Tri Dao.
\newblock Mamba: Linear-time sequence modeling with selective state spaces.
\newblock \emph{ArXiV Preprint}, 2023.
\bibitem[Gu et~al.(2021)Gu, Johnson, Goel, Saab, Dao, Rudra, and Ré]{gu2021combining}
Albert Gu, Isys Johnson, Karan Goel, Khaled Saab, Tri Dao, Atri Rudra, and Christopher Ré.
\newblock Combining recurrent, convolutional, and continuous-time models with linear state-space layers.
\newblock In \emph{Advances in Neural Processing Information Systems}, 2021.
\bibitem[Gu et~al.(2022)Gu, Goel, and Ré]{gu2022s4}
Albert Gu, Karan Goel, and Christopher Ré.
\newblock Efficiently modeling long sequences with structured state spaces.
\newblock In \emph{International Conference on Learning Representations}, 2022.
\bibitem[Guo et~al.(2024)Guo, Xie, Zhang, and Qian]{guo2024causalitygcn}
Rui Guo, Zheng Xie, Chencheng Zhang, and Xiaohua Qian.
\newblock Causality-enhanced multiple instance learning with graph convolutional networks for parkinsonian freezing-of-gait assessment.
\newblock \emph{IEEE Transactions on Image Processing}, 33:\penalty0 3991--4001, 2024.
\bibitem[Guo et~al.(2022)Guo, Yang, Liu, Chen, and Yang]{guo2022parkinsongaitsurvey}
Yao Guo, Jianxin Yang, Yuxuan Liu, Xun Chen, and Guang-Zhong Yang.
\newblock Detection and assessment of parkinson's disease based on gait analysis: A survey.
\newblock \emph{Frontiers in Aging Neuroscience}, 14, 2022.
\bibitem[Guptå et~al.(2022)Guptå, Gu, and Berant]{gupta2022diagonalssm}
Ankit Guptå, Albert Gu, and Jonathan Berant.
\newblock Diagonal state spaces are as effective as structured state spaces.
\newblock In \emph{Advances in Neural Processing Information Systems}, 2022.
\bibitem[Hasani et~al.(2023)Hasani, Lechner, Wang, Chahine, Amini, and Rus]{hasani2022liquidssm}
Ramin Hasani, Mathias Lechner, Tsun-Hsuan Wang, Makram Chahine, Alexander Amini, and Daniela Rus.
\newblock Liquid structural state-space models.
\newblock In \emph{International Conference on Learning Representations}, 2023.
\bibitem[Hou et~al.(2022)Hou, Liu, Cen, Dong, Yang, Wang, and Tang]{hou2022graphmae}
Zhenyu Hou, Xiao Liu, Yukuo Cen, Yuxiao Dong, Hongxia Yang, Chunjie Wang, and Jie Tang.
\newblock Graphmae: Self-supervised masked graph autoencoders.
\newblock In \emph{Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining}, pages 594--604. Association for Computing Machinery, 2022.
\bibitem[Hu et~al.(2019)Hu, Shen, Albanie, Sun, and Wu]{hu2018squeeze}
Jie Hu, Li Shen, Samuel Albanie, Gang Sun, and Enhua Wu.
\newblock Squeeze-and-excitation networks.
\newblock \emph{IEEE Transaction on Pattern Analysis and Machine Intelligence}, 2019.
\bibitem[Kim et~al.(2022)Kim, Chang, Kim, and Choi]{kim2022globallocaltransformer}
Boeun Kim, Hyung~Jin Chang, Jungho Kim, and Jin~Young Choi.
\newblock Global-local motion transformer for unsupervised skeleton-based action learning.
\newblock In \emph{European Conference on Computer Vision}, 2022.
\bibitem[Kour et~al.(2024)Kour, Gupta, and Arora]{kour2024koanmpddataset}
Navleen Kour, Sunanda Gupta, and Sakshi Arora.
\newblock A vision-based hybrid ensemble learning approach for classification of gait disorders.
\newblock \emph{Multimedia Tools and Applications}, 2024.
\bibitem[Lee et~al.(2023{\natexlab{a}})Lee, Lee, Cho, Woo, Jang, and Lee]{lee2023stcnet}
Jungho Lee, Minhyeok Lee, Suhwan Cho, Sungmin Woo, Sungjun Jang, and Sangyoun Lee.
\newblock Leveraging spatio-temporal dependency for skeleton-based action recognition.
\newblock In \emph{IEEE/CVF International Conference on Computer Vision}, pages 10221--10230, 2023{\natexlab{a}}.
\bibitem[Lee et~al.(2023{\natexlab{b}})Lee, Lee, Lee, and Lee]{lee2023hdgcn}
Jungho Lee, Minhyeok Lee, Dogyoon Lee, and Sangyoun Lee.
\newblock Hierarchically decomposed graph convolutional networks for skeleton-based action recognition.
\newblock In \emph{IEEE/CVF International Conference on Computer Vision}, pages 10410--10419, 2023{\natexlab{b}}.
\bibitem[Li and Chen(2024)]{liu2024spmamba}
Kai Li and Guo Chen.
\newblock Spmamba: State-space model is all you need in speech separation, 2024.
\bibitem[Liang et~al.(2024)Liang, Zhou, Xu, Zhu, Zou, Ye, Tan, and Bai]{liang2024pointmamba}
Dingkang Liang, Xin Zhou, Wei Xu, Xingkui Zhu, Zhikang Zou, Xiaoqing Ye, Xiao Tan, and Xiang Bai.
\newblock Pointmamba: A simple state space model for point cloud analysis, 2024.
\bibitem[Lin et~al.(2023)Lin, Zhang, and Liu]{lin2023actionlet}
Lilang Lin, Jiahang Zhang, and Jiaying Liu.
\newblock Actionlet-dependent contrastive learning for unsupervised skeleton-based action recognition.
\newblock In \emph{IEEE/CVF International Conference on Computer Vision and Pattern Recognition}, 2023.
\bibitem[Liu et~al.(2020)Liu, Shahroudy, Perez, Wang, Duan, and Kot]{liu2020ntu120}
Jun Liu, Amir Shahroudy, Mauricio Perez, Gang Wang, Ling-Yu Duan, and Alex~C. Kot.
\newblock Ntu rgb+d 120: A large-scale benchmark for 3d human activity understanding.
\newblock \emph{IEEE Transactions on Pattern Analysis and Machine Intelligence}, 42:\penalty0 2684--2701, 2020.
\bibitem[Liu et~al.(2024)Liu, Chen, and Liu]{liu2024mmcolearning}
Jinfu Liu, Chen Chen, and Mengyuan Liu.
\newblock Multi-modality co-learning for efficient skeleton-based action recognition.
\newblock In \emph{ACM Multimedia}, 2024.
\bibitem[Ma et~al.(2024{\natexlab{a}})Ma, Li, and Wang]{ma2024umamba}
Jun Ma, Feifei Li, and Bo Wang.
\newblock U-mamba: Enhancing long-range dependency for biomedical image segmentation, 2024{\natexlab{a}}.
\bibitem[Ma et~al.(2024{\natexlab{b}})Ma, Huo, Liu, Zhao, Wang, and Xu]{ma2024twintower}
Lan Ma, Hua Huo, Wei Liu, Changwei Zhao, Jinxuan Wang, and Ningya Xu.
\newblock Twin-tower transformer network for skeleton-based parkinson’s disease early detection.
\newblock \emph{Complex and Intelligent Systems}, 2024{\natexlab{b}}.
\bibitem[Myung et~al.(2024)Myung, Su, Xue, and Wang]{myung2024degcn}
Woomin Myung, Nan Su, Jing-Hao Xue, and Guijin Wang.
\newblock Degcn: Deformable graph convolutional networks for skeleton-based action recognition.
\newblock \emph{IEEE Transactions on Image Processing}, 33:\penalty0 2477--2490, 2024.
\bibitem[Naseem et~al.(2024)Naseem, Seo, Kim, and Lee]{naseem2024gaitfootpressureskeleton}
Muhammad~Tahir Naseem, Haneol Seo, Na-Hyun Kim, and Chan-Su Lee.
\newblock Pathological gait classification using early and late fusion of foot pressure and skeleton data.
\newblock \emph{Applied Sciences}, 14:\penalty0 558, 2024.
\bibitem[Nguyen et~al.(2022)Nguyen, Goel, Gu, Downs, Shah, Dao, Baccus, and Ré]{nguyen2022s4nd}
Eric Nguyen, Karan Goel, Albert Gu, Gordon~W Downs, Preey Shah, Tri Dao, Stephen~A Baccus, and Christopher Ré.
\newblock S4nd: Modeling images and videos as multidimensional signals using state spaces.
\newblock In \emph{Advances in Neural Processing Information Systems}, 2022.
\bibitem[Oraki et~al.(2024)Oraki, Zhuang, and Liang]{oraki2024lortsar}
Soroush Oraki, Harry Zhuang, and Jie Liang.
\newblock Lortsar: Low-rank transformer for skeleton-based action recognition, 2024.
\bibitem[Pang et~al.(2022)Pang, Ke, Rahmani, Bailey, and Liu]{pang2022igformer}
Yunsheng Pang, Qiuhong Ke, Hossein Rahmani, James Bailey, and Jun Liu.
\newblock Igformer: Interaction graph transformer for skeleton-based human interaction recognition.
\newblock In \emph{European Conference on Computer Vision}, 2022.
\bibitem[Patro and Agneeswaran(2024)]{patro2024simba}
Badri~N. Patro and Vijay~S. Agneeswaran.
\newblock Simba: Simplified mamba-based architecture for vision and multivariate time series.
\newblock \emph{ArXiV Preprint}, 2024.
\bibitem[Peng et~al.(2020)Peng, Hong, Chen, and Zhao]{peng2020gnnsearch}
Wei Peng, Xiaopeng Hong, Haoyu Chen, and Guoying Zhao.
\newblock Learning graph convolutional network for skeleton-based human action recognition by neural searching.
\newblock In \emph{Proceedings of the AAAI Conference on Artificial Intelligence}, pages 2669--2676, 2020.
\bibitem[Plizzari et~al.(2021)Plizzari, Cannici, and Matteucci]{plizzari2021sttr}
Chiara Plizzari, Marco Cannici, and Matteo Matteucci.
\newblock Spatial temporal transformer network for skeleton-based action recognition.
\newblock In \emph{International Conference on Pattern Recognition}, pages 694--701, 2021.
\bibitem[Serrao et~al.(2016)Serrao, Rinaldi, Ranavolo, Lacquaniti, Martino, Leonardi, Conte, Varrecchia, Draicchio, Coppola, Casali, and Pierelli]{Serrao2016}
Mariano Serrao, Martina Rinaldi, Alberto Ranavolo, Francesco Lacquaniti, Giovanni Martino, Luca Leonardi, Carmela Conte, Tiwana Varrecchia, Francesco Draicchio, Gianluca Coppola, Carlo Casali, and Francesco Pierelli.
\newblock Gait patterns in patients with hereditary spastic paraparesis.
\newblock \emph{PLOS ONE}, 11\penalty0 (10):\penalty0 e0164623, 2016.
\bibitem[Shahroudy et~al.(2016)Shahroudy, Liu, Ng, and Wang]{shahroudy2016ntu60}
Amir Shahroudy, Jun Liu, Tian-Tsong Ng, and Gang Wang.
\newblock Ntu rgb+d: A large scale dataset for 3d human activity analysis.
\newblock In \emph{IEEE International Conference on Computer Vision and Pattern Recognition}, 2016.
\bibitem[Shaker et~al.(2024)Shaker, Wasim, Khan, Gall, and Khan]{shaker2024groupmamba}
Abdelrahman Shaker, Syed~Talal Wasim, Salman Khan, Juergen Gall, and Fahad~Shahbaz Khan.
\newblock Groupmamba: Parameter-efficient and accurate group visual state space model, 2024.
\bibitem[Shi et~al.(2019{\natexlab{a}})Shi, Zhang, Cheng, and Lu]{shi2019agcn}
Lei Shi, Yifan Zhang, Jian Cheng, and Hanqing Lu.
\newblock Two-stream adaptive graph convolutional networks for skeleton-based action recognition.
\newblock In \emph{IEEE/CVF International Conference on Computer Vision and Pattern Recognition}, 2019{\natexlab{a}}.
\bibitem[Shi et~al.(2019{\natexlab{b}})Shi, Zhang, Cheng, and Lu]{shi2019dgnn}
Lei Shi, Yifan Zhang, Jian Cheng, and Hanqing Lu.
\newblock Skeleton-based action recognition with directed graph neural networks.
\newblock In \emph{IEEE/CVF International Conference on Computer Vision and Pattern Recognition}, 2019{\natexlab{b}}.
\bibitem[Shi et~al.(2020{\natexlab{a}})Shi, Zhang, Cheng, and Lu]{shi2020dstanet}
Lei Shi, Yifan Zhang, Jian Cheng, and Hanqing Lu.
\newblock Decoupled spatial-temporal attention network for skeleton-based action-gesture recognition.
\newblock In \emph{Asian Conference on Computer Vision}, pages 38--53, 2020{\natexlab{a}}.
\bibitem[Shi et~al.(2020{\natexlab{b}})Shi, Zhang, Cheng, and Lu]{shi2020msagcn}
Lei Shi, Yifan Zhang, Jian Cheng, and Hanqing Lu.
\newblock Skeleton-based action recognition with multi-stream adaptive graph convolutional networks.
\newblock \emph{IEEE Transactions on Image Processing}, 29:\penalty0 9532--9545, 2020{\natexlab{b}}.
\bibitem[Si et~al.(2019)Si, Chen, Wang, Wang, and Tan]{si2019agclstm}
Chenyang Si, Wentao Chen, Wei Wang, Liang Wang, and Tieniu Tan.
\newblock An attention enhanced graph convolutional lstm network for skeleton-based action recognition.
\newblock In \emph{International Conference on Computer Vision and Pattern Recognition}, 2019.
\bibitem[Smith et~al.(2023)Smith, Warrington, and Linderman]{smith2023simplifiedssm}
Jimmy T.~H. Smith, Andrew Warrington, and Scott~W. Linderman.
\newblock Simplified state space layers for sequence modeling.
\newblock In \emph{International Conference on Learning Representations}, 2023.
\bibitem[Sun et~al.(2019)Sun, Xiao, Liu, and Wang]{sun2019hrnet}
Ke Sun, Bin Xiao, Dong Liu, and Jingdong Wang.
\newblock Deep high-resolution representation learning for human pose estimation.
\newblock In \emph{IEEE/CVF International Conference on Computer Vision and Pattern Recognition}, 2019.
\bibitem[Vaswani et~al.(2017)Vaswani, Shazeer, Parmar, Uszkoreit, Jones, Gomez, Kaiser, and Polosukhin]{vaswani2017attention}
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan~N. Gomez, Lukasz Kaiser, and Illia Polosukhin.
\newblock Attention is all you need.
\newblock In \emph{Advances in Neural Information Processing Systems}, 2017.
\bibitem[Wang et~al.(2014)Wang, Nie, Xia, Wu, and Zhu]{wang2014nwucla}
Jiang Wang, Xiaohan Nie, Yin Xia, Ying Wu, and Song-Chun Zhu.
\newblock Cross-view action modeling, learning and recognition.
\newblock In \emph{IEEE International Conference on Computer Vision and Pattern Recognition}, 2014.
\bibitem[Wang and Koniusz(2023)]{wang20233mformer}
Lei Wang and Piotr Koniusz.
\newblock 3mformer: Multi-order multi-mode transformer for skeletal action recognition.
\newblock In \emph{IEEE/CVF International Conference on Computer Vision and Pattern Recognition}, 2023.
\bibitem[Wang et~al.(2022)Wang, Lv, Jiang, Liu, Wang, Xu, and Li]{gao2022fgstformer}
Peitao Wang, Pei Lv, Xiaoheng Jiang, Qidong Liu, Pichao Wang, Mingliang Xu, and Wanqing Li.
\newblock Focal and global spatial-temporal transformer for skeleton-based action recognition.
\newblock In \emph{Asian Conference on Computer Vision}, 2022.
\bibitem[Wang et~al.(2023)Wang, Xu, and Mu]{wang2023koopman}
Xinghan Wang, Xin Xu, and Yadong Mu.
\newblock Neural koopman pooling: Control-inspired temporal dynamics encoding for skeleton-based action recognition.
\newblock In \emph{IEEE Computer Society Conference on Computer Vision and Pattern Recognition}, pages 10597--10607, 2023.
\bibitem[Wen et~al.(2023)Wen, Tang, Pang, Ding, and Liu]{wen2023istanet}
Yuhang Wen, Zixuan Tang, Yunsheng Pang, Beichen Ding, and Mengyuan Liu.
\newblock Interactive spatiotemporal token attention network for skeleton-based general interactive action recognition.
\newblock In \emph{EEE/RSJ International Conference on Intelligent Robots and Systems}, 2023.
\bibitem[Wu et~al.(2024)Wu, Zheng, Yang, Chen, Das, and Lu]{wu2024freqaware}
Wenhan Wu, Ce Zheng, Zihao Yang, Chen Chen, Srijan Das, and Aidong Lu.
\newblock Frequency guidance matters: Skeletal action recognition by frequency-aware mixed transformer.
\newblock In \emph{ACM Multimedia}, 2024.
\bibitem[Xiang et~al.(2023)Xiang, Li, Zhou, Wang, and Zhang]{xiang2023lst}
Wangmeng Xiang, Chao Li, Yuxuan Zhou, Biao Wang, and Lei Zhang.
\newblock Generative action description prompts for skeleton-based action recognition.
\newblock In \emph{International Conference on Computer Vision}, pages 10242--10251, 2023.
\bibitem[Xu et~al.(2022)Xu, Ye, Zhong, and Xie]{xu2022tacnn}
Kailin Xu, Fanfan Ye, Qiaoyong Zhong, and Di Xie.
\newblock Topology-aware convolutional neural network for efficient skeleton-based action recognition.
\newblock In \emph{AAAI Conference on Artificial Intelligence}, 2022.
\bibitem[Xu et~al.(2024)Xu, Yang, Wang, Cai, Du, and Chen]{xu2024mambasurvey}
Rui Xu, Shu Yang, Yihui Wang, Yu Cai, Bo Du, and Hao Chen.
\newblock Visual mamba: A survey and new outlooks, 2024.
\bibitem[Yan et~al.(2018)Yan, Xiong, and Lin]{yan2018stgcn}
Sijie Yan, Yuanjun Xiong, and Dahua Lin.
\newblock Spatial temporal graph convolutional networks for skeleton-based action recognition.
\newblock In \emph{AAAI Conference on Artificial Intelligence}, 2018.
\bibitem[Yang et~al.(2024)Yang, Xing, Yu, Huang, Fu, and Zhu]{yang2024vivim}
Yijun Yang, Zhaohu Xing, Lequan Yu, Chunwang Huang, Huazhu Fu, and Lei Zhu.
\newblock Vivim: a video vision mamba for medical video segmentation, 2024.
\bibitem[Ye et~al.(2020)Ye, Pu, Zhong, Li, Xie, and Tang]{ye2020dyngcn}
Fanfan Ye, Shiliang Pu, Qiaoyong Zhong, Chao Li, Di Xie, and Huiming Tang.
\newblock Dynamic gcn: Context-enriched topology learning for skeleton-based action recognition.
\newblock In \emph{Proceedings of the 28th ACM International Conference on Multimedia}, pages 55--63. ACM, 2020.
\bibitem[Zhang et~al.(2024{\natexlab{a}})Zhang, Fan, He, Lei, Zhang, and Zhang]{zhang2024voxelmamba}
Guowen Zhang, Lue Fan, Chenhang He, Zhen Lei, Zhaoxiang Zhang, and Lei Zhang.
\newblock Voxel mamba: Group-free state space models for point cloud based 3d object detection, 2024{\natexlab{a}}.
\bibitem[Zhang et~al.(2023)Zhang, Lim, Kim, Hur, and Chung]{zhang2023wmstgcn}
Jieming Zhang, Jongmin Lim, Moon~Hyun Kim, Sungwook Hur, and Tai~Myoung Chung.
\newblock Wm–stgcn: A novel spatiotemporal modeling method for parkinsonian gait recognition.
\newblock \emph{Sensors}, 23, 2023.
\bibitem[Zhang et~al.(2020{\natexlab{a}})Zhang, Lan, Zeng, Xing, Xue, and Zheng]{zhang2020sgn}
Pengfei Zhang, Cuiling Lan, Wenjun Zeng, Junliang Xing, Jianru Xue, and Nanning Zheng.
\newblock Semantics-guided neural networks for efficient skeleton-based human action recognition.
\newblock In \emph{IEEE/CVF International Conference on Computer Vision and Pattern Recognition}, 2020{\natexlab{a}}.
\bibitem[Zhang et~al.(2020{\natexlab{b}})Zhang, Xu, and Tao]{zhang2020contextawaregcn}
Xikun Zhang, Chang Xu, and Dacheng Tao.
\newblock Context aware graph convolution for skeleton-based action recognition.
\newblock In \emph{IEEE/CVF International Conference on Computer Vision and Pattern Recognition}, 2020{\natexlab{b}}.
\bibitem[Zhang et~al.(2021)Zhang, Wu, Li, Duan, and Gan]{zhang2021stst}
Yuhan Zhang, Bo Wu, Wen Li, Lixin Duan, and Chuang Gan.
\newblock Stst: Spatial-temporal specialized transformer for skeleton-based action recognition.
\newblock In \emph{ACM International Conference on Multimedia}, pages 3229--3237, 2021.
\bibitem[Zhang et~al.(2024{\natexlab{b}})Zhang, Liu, Reid, Hartley, Zhuang, and Tang]{zhang2024motionmamba}
Zeyu Zhang, Akide Liu, Ian Reid, Richard Hartley, Bohan Zhuang, and Hao Tang.
\newblock Motion mamba: Efficient and long sequence motion generation, 2024{\natexlab{b}}.
\bibitem[Zhou et~al.(2023)Zhou, Liu, and Wang]{zhou2023}
Huanyu Zhou, Qingjie Liu, and Yunhong Wang.
\newblock Learning discriminative representations for skeleton based action recognition.
\newblock In \emph{IEEE/CVF International Conference on Computer Vision and Pattern Recognition}, 2023.
\bibitem[Zhou et~al.(2022)Zhou, Cheng, Li, Fang, Geng, Xie, and Keuper]{zhou2022hyperformer}
Yuxuan Zhou, Zhi-Qi Cheng, Chao Li, Yanwen Fang, Yifeng Geng, Xuansong Xie, and Margret Keuper.
\newblock Hypergraph transformer for skeleton-based action recognition, 2022.
\bibitem[Zhou et~al.(2024)Zhou, Yan, Cheng, Yan, Dai, and Hua]{zhou2024blockgcn}
Yuxuan Zhou, Xudong Yan, Zhi-Qi Cheng, Yan Yan, Qi Dai, and Xian-Sheng Hua.
\newblock Blockgcn: Redefine topology awareness for skeleton-based action recognition.
\newblock In \emph{IEEE/CVF International Conference on Computer Vision and Pattern Recognition}, 2024.
\bibitem[Zhu et~al.(2023)Zhu, Ke, Gong, and Bailey]{zhu2023adaptive}
Anqi Zhu, Qiuhong Ke, Mingming Gong, and James Bailey.
\newblock Adaptive local-component-aware graph convolutional network for one-shot skeleton-based action recognition.
\newblock In \emph{IEEE/CVF Winter Applications of Computer Vision}, 2023.
\end{thebibliography}