Versatile Loco-Manipulation through Flexible Interlimb Coordination
Extracted source text · arXiv 2506.07876
\documentclass{article}
\usepackage{amssymb}
\input{notion_marcos}
% \usepackage{corl_2025} % Use this for the initial submission.
% \usepackage[final]{corl_2025} % Uncomment for the camera-ready ``final'' version.
\usepackage[preprint]{corl_2025} % Uncomment for pre-prints (\eg, arxiv); This is like ``final'', but will remove the CORL footnote.
% \title{\algabrvname: Flexible Interface for Loco-Manipulation}
% \title{Contact-Adaptive Loco-Manipulation through Flexible Interlimb Coordination}
\title{Versatile Loco-Manipulation through \\ Flexible Interlimb Coordination}
% The \author macro works with any number of authors. There are two
% commands used to separate the names and addresses of multiple
% authors: \And and \AND.
%
% Using \And between authors leaves it to LaTeX to determine where to
% break the lines. Using \AND forces a line break at that point. So,
% if LaTeX puts 3 of 4 authors names on the first line, and the last
% on the second line, try using \AND instead of \And before the third
% author name.
% NOTE: authors will be visible only in the camera-ready and preprint versions (i.e., when using the option 'final' or 'preprint').
% For the initial submission the authors will be anonymized.
\author{
\textbf{Xinghao Zhu}$^{\ast,1}$, \textbf{Yuxin Chen}$^{\ast,1,2}$, \textbf{Lingfeng Sun}$^{\ast,1}$, \textbf{Farzad Niroui}$^1$ \\
\textbf{Simon Le Cleac'h}$^1$, \textbf{Jiuguang Wang}$^1$, \textbf{Kuan Fang}$^{1,3}$ \\
$^{\ast}$Denotes equal contribution \\
$^1$\textit{RAI Institute} \quad $^2$\textit{University of California, Berkeley} \quad $^3$\textit{Cornell University}
}
\begin{document}
\maketitle
%===============================================================================
\vspace{-20pt}
\begin{figure}[h]
\centering
\includegraphics[width=\linewidth]{figures/teaser.pdf}
% \includegraphics[width=\linewidth]{figures/teaser_v2.png}
\caption{
% \textbf{\algname~(\algabrvname).}
% \textbf{Interlimb coordination for loco-manipulation.}
% We present \algabrvname, a learning-based framework that enables flexible coordination of limbs for versatile loco-manipulation involving rich contacts in unstructured environments.
% \algabrvname enables a arm-mounted quadruped robot to
% dynamically assign the role to each limb, such as manipulating the object with both the arm (\textcolor{LimeGreen}{\textit{green}}) and a foot (\textcolor{OrangeRed}{\textit{red}}) as end-effectors, while performing stable gaits with the remaining legs as support limbs (\textcolor{Orchid}{\textit{purple}}).
% The effectiveness of \algabrvname is validated on a variety of tasks specified by different user inputs such as target trajectories, contact points, and language instructions.
We present \algabrvname, a learning-based approach that enables flexible interlimb coordination for versatile loco-manipulation in unstructured environments. \algabrvname controls an arm-mounted quadruped robot to dynamically assign limb for either locomotion or manipulation during task execution, \eg, simultaneously using an arm ({\color{relic_green}\textit{green}}) and a selected leg ({\color{relic_red}\textit{red}}) to jointly reach manipulation targets while maintaining stable mobility with the remaining legs ({\color{relic_purple}\textit{purple}}). The effectiveness of our approach is demonstrated across diverse and complex tasks specified by various user input modalities, including direct target specification, contact points, and natural language instructions.
}
\label{fig:teaser}
\vspace{-1.1em}
\end{figure}
\begin{abstract}
The ability to flexibly leverage limbs for loco-manipulation is essential for enabling autonomous robots to operate in unstructured environments. Yet, prior work on loco-manipulation is often constrained to specific tasks or predetermined limb configurations. In this work, we present \algname (\algabrvname), an approach that enables versatile loco-manipulation through flexible interlimb coordination. The key to our approach is an adaptive controller that seamlessly bridges the execution of manipulation motions and the generation of stable gaits based on task demands. Through the interplay between two controller modules, \algabrvname dynamically assigns each limb for manipulation or locomotion and robustly coordinates them to achieve the task success.
Using efficient reinforcement learning in simulation, \algabrvname learns to perform stable gaits in accordance with the manipulation goals in the real world.
To solve diverse and complex tasks, we further propose to interface the learned controller with different types of task specifications, including target trajectories, contact points, and natural language instructions. Evaluated on 12 real-world tasks that require diverse and complex coordination patterns, \algabrvname demonstrates its versatility and robustness by achieving a success rate of 78.9\% on average.
Videos and code can be found at \href{https://relic-locoman.rai-inst.com/}{\textcolor{magenta}{https://relic-locoman.rai-inst.com/}}.
% Legged robots possess significant potential for real-world applications requiring coordinated locomotion and manipulation. Current systems, however, typically treat locomotion and manipulation capabilities separately, limiting their ability to coordinate limbs for complex multi-contact interactions. We present \algabrvname, a hierarchical framework for whole-body loco-manipulation that unifies high-level contact reasoning with low-level control. \algabrvname decomposes tasks into contact-oriented sub-goals to specify contact targets and employs a limb-agnostic reinforcement learning policy to execute these goals. The policy seamlessly transitions between four-leg and three-leg locomotion while tracking arm and leg trajectories. Extensive experimental validation on an arm-equipped quadruped demonstrates \algabrvname's capabilities across 12 challenging scenarios spanning three categories: mobile limb coordination, stationary limb coordination, and foot-assisted manipulation. The results show consistent generalization across diverse contact configurations, advancing adaptive loco-manipulation in unstructured environments.
\end{abstract}
% Two or three meaningful keywords should be added here
\keywords{Loco-Manipulation, Whole-Body Control, Reinforcement Learning}
%===============================================================================
\section{Introduction}
\label{sec:introduction}
The ability to combine locomotion and manipulation, commonly referred to as \textit{loco-manipulation}, is essential to achieving robot autonomy in unstructured environments~\citep{rehman2016mobilemanipulator}.
As tasks grow in diversity and complexity, robots must perform versatile interactions with both the terrain and external objects, coordinating limb movements in concert.
Despite significant advances in locomotion and manipulation individually, seamlessly integrating the two capabilities remains a major open challenge~\citep{sleiman2023versatile, gu2025humanoidlocomotionmanipulationcurrent}.
Consider, for example, an arm-mounted quadruped robot tasked with transporting a large yoga ball across a room, as shown in \Cref{fig:teaser}.
Success of this task demands coordinating the arm and a leg to grasp, lift, and balance the object, while maintaining stable gaits to move around with the remaining support legs.
Different tasks impose different contact and movement requirements, necessitating adaptive whole-body control strategies that can flexibly allocate actuation across limbs.
In light of this need, prior work has explored a range of decomposition and coordination strategies. A common approach partitions control among limb groups, typically treating arms and legs separately, based on task-specific heuristics~\citep{zimmermann2021go,hooks2020alphred}. While effective in constrained settings such as mobile pick-and-place, these methods often depend heavily on manual design and struggle to generalize to tasks requiring dynamic assignment of limb functions. More recent efforts have pursued whole-body control, spanning both model-based and learning-based approaches~\citep{bellicoso2019alma, sleiman2023versatile, fu2023deep}. Despite encouraging results, these methods are often confined to predefined task sets and fixed limb roles. For more complex settings, jointly optimizing for both locomotion and manipulation within a unified framework remains a fundamental challenge in practice.
To this end, we present \textbf{Re}inforcement \textbf{L}earning for \textbf{I}nterlimb \textbf{C}oordination (\textbf{\algabrvname}), an approach for solving versatile loco-manipulation in unstructured environments. At the core of \algabrvname is an adaptive controller that bridges manipulation success with locomotion stability under dynamic assignments of limb functions. Instead of solving both objectives holistically or relying on fixed decomposition heuristics, our method decouples loco-manipulation into two interconnected subproblems, robustly generating manipulation behaviors and maintaining stable gaits based on task demands.
% The two subproblems are handled by two separate controllers that communicate via a mask interface, which dynamically assigns limbs to either manipulation or locomotion based on task demands.
We train the locomotion controller entirely in simulation using an efficient reinforcement learning pipeline and then transfer it to the real world through motor calibration. Built on top of this adaptive controller, our approach supports flexible task specification via high-level task interfaces, including direct targets, contact points, and free-form language instructions. As illustrated in \Cref{fig:teaser}, \algabrvname enables a broad range of loco-manipulation tasks requiring seamless interlimb coordination. Deployed on an arm-mounted Boston Dynamics Spot, \algabrvname demonstrates robust performance across 12 diverse tasks involving mobile interlimb manipulation, stationary interlimb coordination, and foot-assisted manipulation. We achieve an overall success rate of 78.9\% across the three different types of task specifications.
%===============================================================================
\begin{figure}[t]
\centering
\includegraphics[width=\linewidth]{figures/pipeline.pdf}
% \vspace{2px}
\caption{\textbf{Overview of \algabrvname.} Based on various types of task specifications, \algabrvname enables the robot to perform versatile loco-manipulation.
Conditioned on the dynamic assignment of limb functions, the \algabrvname controller generates the actions through the interplay between a model-based module that prioritizes task success, and an RL policy that robustly maintain stable gaits in accordance with the manipulation behaviors.
This design decouples the two challenging subproblems without relying on rigid heuristics or sacrificing inter-module coordination.
% The \algabrvname controller dynamically bridges manipulation and locomotion using a model-based module and a trained RL policy communicated via a mask interface, enabling solving the manipulation task while maintaining stable gaits.
%
% decomposes tasks into a hierarchical structure of task-level and command-level. On task-level, limbs and targets are specified via 3D point cloud by user annotation or 2D images by VLM. On command-level, the low-level RL controller takes in proprioceptive robot state and output joint-level actions conditioned on the arm, leg, and body targets.
% \kf{This needs further polishing.}
}
\label{fig:overview}
\end{figure}
\section{Related Work}
\label{sec:related_work}
%% Paragraph 1: Loco-manipulation
%% - Joint locomotion and manipulation has been widely investigated in robotics using hardware such as wheeled manipulators, humanoid robots, quadruped robots.
%% - Model-based approach,
%% - Learning-based approach,
%% - Through a task-oriented design, our approach decouples the locomotion and manipulation problems, using a model-based controller to solve the manipulation task while training an RL policy to generate stable gaits.
%% - Most of these approaches are constrained to specific tasks or predefined roles for each limb, ...
%% - In contrast, our approach ...
%% Paragraph 2: Loco-manipultion with interlimb coordination.
%% - Interlimb coordination for humans and animals (cite 1-3 papers from non-robotics area)
%% - Although coordination of multiple arms or legs has been studied in robotic manipulation and locomotion individually, interlimb coordination for loco-manipulation remains an open challenge.
%% - Works using legs as manipulators.
%% - Recently \citet{sleiman2023versatile, sleiman2024guided}, automatically discovers whole-body for loco-manipulation tasks involving rich contacts in pre-modeled environments.
%% - However, this line of work does not allow dynamic allocation of limb roles and assumes known object models and affordances.
%% -
Joint locomotion and manipulation has been widely studied in robotics, with applications across diverse hardware platforms including wheeled mobile manipulators~\citep{zhang2024catch,bajracharya2024demonstrating,fu2024mobile,hu2023causal,kindle2020whole,lew2023robotic,sun2022fully,yang2024harmonic,wu2023tidybot}, humanoid robots~\citep{he2024omnih2o,fu2024humanplus,ze2024generalizable,lu2024mobile}, and legged systems~\citep{bellicoso2019alma, ma2022combining, ewen2021generating, yokoyama2023asc,ferrolho2023roloma, zimmermann2021go, arcari2023bayesian, zhang2024gamma, jeon2023learning, fu2023deep, ha2024umi, liu2024visual}.
Model-based approaches typically rely on whole-body control frameworks and trajectory optimization, leveraging accurate dynamics models and structured task representations~\citep{bellicoso2019alma, ma2022combining, ewen2021generating, yokoyama2023asc}. Assuming detailed knowledge of the scene, object geometry, and contact conditions~\citep{ferrolho2023roloma, zimmermann2021go, arcari2023bayesian, zhang2024gamma}, most of these approaches operate in controlled settings and often struggle to scale to unstructured environments.
On the other hand, learning-based methods offer improved adaptability and reduced engineering overhead by leveraging large-scale, high-quality data~\citep{jeon2023learning, fu2023deep, ha2024umi, liu2024visual}. However, most are limited to specific tasks or assume fixed limb roles, due to the complex nature of learning such capabilities and the limited data~\citep{sleiman2024guided, pan2024roboduet}.
Our approach follows a task-oriented design that decouples the loco-manipulation problem solved through an integration of a model-based controller and trained RL policy. This separation enables more flexible reuse of learned behaviors across tasks while supporting dynamic reallocation of limb roles in response to task demands.
Interlimb coordination is a hallmark of natural locomotion and dexterous behavior in animals and humans~\citep{fuchs1994theoretical, wannier2001arm, donker2001coordination}. In robotics, coordinated use of multiple limbs has been explored individually for multi-legged locomotion~\citep{aoi2017adaptive} and bimanual manipulation~\citep{tarn1986coordinated, lin2024twisting}.
Recent work has investigated using legs as manipulators to extend task versatility~\citep{wolfslag2020optimisation, lin2024locoman, arm2024pedipulate, he2024learning, cheng2023legs}.
However, these systems often rely on predefined contact sequences with explicit mode switching, sometimes requiring additional actuation hardware.
Notably, \citet{sleiman2023versatile, sleiman2024guided} propose whole-body planning for contact-rich tasks using pre-modeled scenes, discovering feasible contact modes offline.
While effective in pre-defined settings, these methods usually assume static assignments of limb functions and known objects, limiting their applicability in new tasks or unseen environments.
In contrast, our approach enables flexible interlimb coordination, allowing each limb to dynamically alternate between locomotion and manipulation based on the evolving requirement for diverse and complex tasks.
% \deprecated{
% Loco-manipulation has been widely explored through both model-based and learning-based approaches.
% Model-based methods typically rely on well-engineered whole-body controllers combined with trajectory optimization~\citep{}.
% Their successful deployment often depends on highly accurate models and minimal external disturbances. These systems are either teleoperated~\cite{bellicoso2019alma, ma2022combining} or used autonomously in narrowly defined, carefully designed scenarios~\cite{ewen2021generating, sleiman2023versatile, zhang2024gamma, yokoyama2023asc, arcari2023bayesian}, often requiring detailed knowledge of the scene and objects~\cite{ferrolho2023roloma, zimmermann2021go}.
% Learning-based methods reduce engineering overhead and offer increased robustness to disturbances. However, they tend to focus on specific, predefined tasks~\cite{jeon2023learning, sleiman2024guided} or show limited generalization in simple arm-based grasping tasks~\cite{fu2023deep, liu2024visual, ha2024umi, pan2024roboduet}.
% Beyond arm-centric approaches, some recent work investigates using legs as manipulators. However, these typically require additional hardware for non-prehensile manipulation~\cite{lin2024locoman,arm2024pedipulate}, assume a stationary body~\cite{wolfslag2020optimisation}, operate within predefined tasks~\cite{he2024learning, he2025visual}, or depend on explicit mode switching~\cite{cheng2023legs}.
% In contrast, our system uniquely coordinates both arms and legs for multi-contact loco-manipulation with diverse and flexible contact configurations. Such as using the arm and one leg for manipulation while allowing the locomotion ability. \emph{It introduces a general-purpose policy with a versatile interface, capable of handling tasks across different contact modalities}—all without requiring explicit mode transitions.
% }
% \textbf{Physical Grounding Reasoning for Robot Manipulation.} Leveraging foundation models in robotics has become a rapidly growing research direction. Vision-language models (VLMs) have demonstrated strong potential in integrating visual inputs~\cite{radford2021learning,ramesh2021zero,li2022blip,achiam2023gpt,li2023blip,xu2024advancing} to support general-purpose robotic tasks. Within robot manipulation, compact representations such as keypoints are commonly used to capture essential information about objects and their environment~\cite{bahl2023affordances,fang2018demo2vec,fang2023generalization,maitin2010cloth,manuelli2019kpam,miller2011parametrized}. Recent work has shown that abstracting complex tasks into sequences of sub-tasks with explicit contact planning enables strong generalization to open-vocabulary manipulation tasks~\cite{ahn2022can,hu2023look,ren2023robots,huang2023voxposer,huang2024rekep,liu2024moka,nasiriany2024pivot}. Building on this line of research, we show that \algabrvname offers a unique capability to autonomously handle complex multi-contact loco-manipulation. Our system integrates high-level reasoning for contact-oriented task decomposition, generates reactive contact points, and enables closed-loop execution with visual re-planning through prompt-based feedback.
%===============================================================================
% \section{\algname}
\section{Method}
Our goal is to control a legged manipulator to flexibly utilize its limbs to solve manipulation tasks while stably performing locomotion across diverse scenarios.
While our approach is designed to generalize to robots with different arm–leg configurations, our experiments and examples focus on a quadrupedal robot equipped with a single arm.
Concretely, the set of available limbs is denoted as $\Lambda = \{ \textit{Arm}, \textit{FL-Leg}, \textit{FR-Leg}, \textit{HL-Leg}, \textit{HR-Leg} \}$, where $\textit{F}$, $\textit{H}$, $\textit{L}$, and $\textit{R}$ indicate front, hind, left, and right respectively.
% As illustrated in \Cref{fig:teaser}, the robot can select a subset of limbs, such as the arm coupled with one leg, for manipulation, with the remaining limbs to support locomotion.
In contrast to prior work~\cite{sleiman2023versatile}, our approach enables each limb to dynamically switch roles between locomotion and manipulation in response to the task demand.
We present \algname(\algabrvname), an approach for versatile loco-manipulation through flexible interlimb coordination (\Cref{fig:overview}).
In this section, we first provide an overview of our framework for versatile loco-manipulation (\Cref{subsec:overview}).
Next, we propose an adaptive controller by leveraging reinforcement learning (RL) to bridge locomotion and manipulation (\Cref{subsec:rl}). We then explain how to effectively learn a robust and transferable policy from simulation for our proposed controller (\Cref{subsec:sim2real}). Lastly, we describe how the \algabrvname controller can be interfaced with various types of user specifications to perform diverse and complex loco-manipulation tasks (\Cref{subsec:task_interface}).
% To enable flexible interlimb coordination, we propose a hierarchical framework consisting of two key layers: high-level planning (\Cref{subsec:overview}), which determines limb allocation and desired motion, and low-level control (\Cref{subsec:rl}), which translates these motions into motor control signals for execution.
% To generate motion trajectories at the high level, we introduce a contact-based planning approach that supports diverse loco-manipulation tasks. This planner operates via user annotations or VLM-based predictions, as detailed in \Cref{subsec:task_interface}.
% An overview of the complete framework is provided in \Cref{fig:overview}.}
% In this section, we first introduce the hierarchical structure through task-level and command-level decomposition (\Cref{subsec:overview}).
% We then describe the command-level controller design for flexible interlimb coordination (\Cref{subsec:rl}), followed by the sim-to-real techniques that robustify simulation-trained policies(Sec.\Cref{subsec:sim2real}).
% Finally, we present a task-level contact-based planning that supports diverse loco-manipulation tasks through user annotation or VLM prediction(\Cref{subsec:task_interface}).
% An overview of the proposed framework is illustrated in \Cref{fig:overview}.
\subsection{Overview}
\label{subsec:overview}
% Instead of committing to a fixed limb assignment as in prior work~\citep{sleiman2023versatile}, our framework dynamically assigns limb $\lambda \in \Lambda$ to manipulation or locomotion and coordinate their movements during the task execution.
% Based on visual observations and task descriptions, our high-level planner simultaneously determines limb assignments, plans limb motions, and generates reference trajectories for low-level tracking.
% In this section, we will explain how the contact-adaptive manipulation is formulated on the task level and the command level in our hierarchical framework.
We propose to solve versatile loco-manipulation in a hierarchical framework consisting of a \textit{task level} and a \textit{command level}.
The former aims to describe diverse and complex loco-manipulation tasks, as shown in \Cref{fig:execution}, using a unified target-driven representation, while the latter controls the robot to achieve the task-specific targets while maintaining stable mobility through flexible coordination of limbs.
Below, we elaborate on the design and interface for each level.
\textbf{Task level.}
Based on the user description, we assume the task solution can be represented as end-effector targets for designated limbs over time.
At each time step, we use a binary mask $m \in \{0, 1\}^{|\Lambda|}$ to indicate the role for each limb $\lambda \in \Lambda$, with manipulation indicated by $1$ and locomotion by $0$.
Given the assignment mask $m$, the target pose for each limb end-effector designated for manipulation is specified as $\tau^\lambda$, leaving the remaining limbs for locomotion with unspecified targets.
Accordingly, the desired torso target $\tau^\text{torso}$ is determined by the manipulation limb targets via whole-body IK~\citep{Zakka_Mink_Python_inverse_2024}, all together denoted as $\tau$.
Now, the task of the horizon $T$ can be represented as the sequence of $\{ \tau_t, m_t \}$ for $t = 0, ..., T-1$.
In \Cref{subsec:task_interface}, we will explain how such task representation can be computed from the user descriptions of various modalities.
\textbf{Command level.}
Given the representation from the task level, we design the controller to generate motor commands for the robot to perform versatile loco-manipulation as shown in \Cref{fig:overview}.
On the Spot robot, low-level commands include the desired position for each joint.
Next, we will explain how to obtain the controller generating commands based on $\{\tau_t,m_t \}$ at each time step.
% run a target-tracking controller which aim to provide robust control to reach manipulation goals while maintaining stable gaits for locomotion.
% As described in Sec.~\ref{sec:prelim}, the controller get sequence of commands consists of the desired limb targets for manipulation and torso target for locomotion $\xi$ and $m$ from task level planning.
% Given the commands, the controller need to calculate action $a$ indicates the target joint positions for all actuators on the robot.
% \ls{The most challenging part for the low-level controller is that it should be able to track arbitrary combination of targets from the high-level planner: flexible limb selections, manipulation and locomotion goals.}
% In the next subsection, we will explain how our framework effectively design the low-level controller using a combination of learning-based and optimization-based approach.
\subsection{Adaptive Control for Flexible Interlimb Coordination}
% \subsection{Bridging Control for Locomotion and Manipulation}
\label{subsec:rl}
To tackle the substantial challenge of solving whole-body control under dynamic limb assignments, we design the \algabrvname controller to seamlessly bridge locomotion and manipulation.
Unlike prior work that learns a monolithic policy end-to-end~\citep{lin2024locoman, he2024learning, fu2023deep, sleiman2024guided}, our approach generates actions through the interaction of two dedicated controller modules: a manipulation module that prioritizes task success, and a locomotion module that maintains stable gaits in accordance with the manipulation behaviors.
The two modules in the \algabrvname controller communicate through the up-to-date whole-body robot state $s_t$ and the limb assignment $m$ to jointly predict the motor commands as the action $a$.
This design decouples the two challenging subproblems without relying on rigid heuristics or sacrificing inter-module coordination.
According to the different needs for manipulation and locomotion, we adopt different design options for the two controller modules.
Given the target-driven task representation, the manipulation task can be directly solved by a model-based (MB) controller conditioned on $s$, $m$, and $\tau$.
Depending on the task requirements, this can be anything from a standard inverse kinematics solver to a meticulously tuned impedance controller.
As for locomotion, which involves dynamic behaviors with significantly higher demands for robustness and adaptability, we train a policy $\pi(\cdot | s, m)$ through reinforcement learning (RL)~\citep{mittal2023orbit}.
Concretely, we denote the action yielded by the two modules as $a_\text{MB}$ and $a_\text{RL}$, both of which have the same dimensionality with the final action $a$ with only the dimensions for the assigned limbs to be valid.
To this end, the final action is computed as $a = m \circ a_\text{MB} + (1 - m) \circ a_\text{RL}$.
To ensure stable locomotion across different limb coordination patterns, we exert gait regularization during training by leveraging contact-time metrics among feet~\cite{mittal2023orbit,miller2025high}.
Specifically, we enforce trotting gait for quadrupedal locomotion and a three-phase bouncing gait for tripedal locomotion, as visualized in \Cref{fig:gait_switch}.
Compared to phase-based gait regularization~\cite{siekmann2021gait}, the contact-time-based approach is simpler to implement and more stable during training, as it avoids sampling from a state-dependent phase variable.
Implementation details are provided in the Appendix.
% \textbf{Bridging locomotion and manipulation.} With the learned RL policy providing joint controls for locomotion-use legs, we can integrate arbitrary joint controls for the manipulation-use limbs.
% In our framework, we use a simple but effective approach to calculate limb joint targets via model-based inverse kinematics from target motions.
% The two components jointly decide the full low-level control signals $a = [a_\text{MB}, a_\text{RL}]$, where $a_\text{MB}$ and $a_\text{RL}$ are the outputs of the IK and the RL policy, respectively.
% Compared to prior end-to-end approaches~\cite{lin2024locoman,he2024learning}, our formulation improves convergence and ensures accurate manipulation limb tracking with model-based IK. Moreover, exposing the manipulation limb control to higher-level planning enables the system to seamlessly extend to more advanced controllers and planners for manipulation, such as compliant admittance controllers~\cite{zhang2023impedance} and collision-avoidance planners~\cite{fishman2022motionpolicynetworks}.
% \textbf{Reinforcement learning for stable locomotion.}
% We train the policy $\pi_\theta(a_\text{RL} | s, m)$ through reinforcement learning to generate the action $a_\text{RL}$ for locomotion.
% While the policy outputs actions for all legs (fixed action dimension), it masks the manipulation-use leg using $m$, effectively restricting execution to locomotion limbs only.
% To guarantee stable gaits across different limb coordination patterns, we enforce trotting gait for quadrupedal locomotion and a three-phase bouncing gait (120$^\circ$ phase shift) for tripedal locomotion, as visualized in Fig.~\ref{fig:gait_switch}.
% Gait regularization leverages contact-time metrics among feet~\cite{mittal2023orbit}, which measure ground contact duration. Compared to phase-based gait regularization~\cite{siekmann2021gait}, the contact-time-based approach is simpler to implement and more stable during training, as it avoids sampling from a state-dependent phase variable.
% Implementation details and MDP definitions are provided in the Appendix.
% A potential approach for low-level control involves training an end-to-end unified whole-body RL policy $\pi_\theta(a | s, m, \xi)$, where the policy tracks arbitrary motions with manipulation-use limbs while achieving locomotion objectives with the remaining legs ($s$ represents proprioceptive states). However, our experimental results reveal significant challenges in training such a policy end-to-end, as it frequently converges to local minima and requires advantage mixing~\cite{fu2023deep} to decouple manipulation and locomotion behaviours.
% This challenge is further compounded by our requirement for dynamic limb role switching during execution. Most existing methods fail to address this need for real-time gait adaptation when limb functions change.
% We therefore propose an alternative formulation: when the torso pose is predetermined (from~\Cref{subsec:overview}), we can compute manipulation limb positions analytically via IK and reduce the RL policy's scope to only maintain stable gaits with locomotion limbs.
% The whole body control problem becomes more complex since the policy need to switch gait while tracking multiple end-effector targets.\ls{ask xinghao for a best sentence supporting this.}
% and arbitrary configurations for limbs that not used for locomotion.
% \kf{What RL algorithm. Brifly describe it.}
% For four-leg trotting, diagonal feet are constrained to have equal contact times. For three-leg bouncing, contact times are staggered by $\frac{T}{3}$ seconds in a fixed order, where $T$ is the target gait cycle duration.
% \ls{can delete more if no space and put in appendix}
% Additionally, we regularize foot air time, defined as the duration from a foot’s last ground contact, to the contact time of its diagonal counterpart during three-leg walking.
% To train $\pi$, the reward $r$ includes the tracking reward encouraging accurate tracking of body velocity, orientation, joint trajectories, and regularization terms penalizing high joint torques, accelerations, energy usage, abrupt action changes, and undesirable gaits to ensure smooth and efficient motion.
%% KF(4/26): This paragraph still needs a lot work. I will take a pass after Yuxin or Lingfeng.
% As introduced in Section~\ref{sec:prelim}, the RL controller $\pi_{\text{loco}}$ is trained to track the waypoint trajectories generated by the hierarchical decomposition strategy in the command space.
% There are several challenges in training this RL policy as our low-level controller. We want a single policy to perform both unimanual (arm) and bimanual (arm and leg) manipulation tasks while maintaining stable locomotion capability. Therefore the policy need to be stable with different gaits and track not only body but also specific leg joint targets. The sim2real transfer gap is also larger since the contact modes of the robot is also richer compared to locomotion tasks. As a result, we propose several design decisions in training and deployment phase, including loco-manipulation command and action space designs, natural gait regularization, and real-world motor calibration.
% \textbf{Adaptive RL controller.}
% The policy $\pi_\text{loco}$ generates leg control signals $q_\text{loco}$ to track desired base velocity and attitude.
% For the designated tracking leg, we employ a residual control scheme to enhance performance:
% $a_\text{leg} = (1-m)\circ q_\text{loco} + m \circ q^c_\text{leg}$, where $m$ is an element-wise indicator function denoting whether a joint is designated for tracking leg joints (\textit{i.e.}, $q^c_\text{leg} \neq 0$). Compared to prior approaches that directly use policy outputs for both locomotion and tracking~\cite{lin2024locoman,he2024learning}, this formulation provides improved tracking precision, system stability, and faster convergence. It also exposes joint-level control to higher-level planning, enabling seamless extension to manipulation tasks.
% For arm control, the desired arm joints $q^c_\text{arm}$ are concatenated with the leg control signal to form the complete actuator command.
% \textbf{Stable gait generation.}
% Another thing we found useful for training policies with smooth gain transition is the natural gait regularization:
% %% KF(04/19): The rest of this paragraph seems a bit like an implemention details without much insights. Some thoughts for improvement: Better motivate this paragraph by clearly explaining the challenges and what the prior work does, start with a high-level math formulation in the context of policy learning, and only concisely talk about the specific regularization pattern for each leg tracking mode at the end.
% we enforce trotting during four-leg walking, and encourage a three-phase bouncing gait with a 120-degree phase shift during three-leg walking, as in Fig.~\ref{fig:gait_switch}. Regularization is based on the contact time between feet~\cite{mittal2023orbit}, which estimates the duration of contact between feet and the ground. For four-leg trotting, diagonal feet are constrained to have equal contact times. For three-leg bouncing, contact times are staggered by $\frac{T}{3}$ seconds in a fixed order, where $T$ is the target gait cycle duration. Additionally, we regularize foot air time, defined as the duration from a foot’s last ground contact, to the contact time of its diagonal counterpart during three-leg walking. Compared to phase-based gait regularization~\cite{siekmann2021gait}, the contact/air time-based approach is simpler to implement and more stable during training, as it avoids sampling from a state-dependent phase variable. Implementation details and timing specifications are provided in Appendix~\ref{app:mdp}.
\begin{figure}[t]
\centering
\includegraphics[width=\linewidth]{figures/execution.pdf}
\caption{\textbf{Loco-Manipulation Tasks with Interlimb Coordination.} \algabrvname is evaluated on 12 real-world tasks designed to test diverse and complex interlimb coordination. The task suite spans three categories: mobile interlimb coordination (\textbf{M}), stationary interlimb coordination (\textbf{S}), and foot-assisted manipulation (\textbf{F}). All tasks can be specified using direct target inputs, with a subset also supports specification via contact points (*) and language instructions (**).}
\label{fig:execution}
\end{figure}
\subsection{Learning Transferrable Policy in Simulation}
\label{subsec:sim2real}
% \textbf{Learning Transferrable Policy in Simulation.}
We leverage IsaacLab~\citep{mittal2023orbit} to scale up reinforcement learning for improving the robustness of the policy $\pi$.
To facilitate sim-to-real transfer, we implement comprehensive domain randomization during training, covering variations in robot dynamics, terrain properties, and external disturbances. Despite these efforts, the dynamic behaviors required for flexible interlimb coordination, particularly during three-legged locomotion, reveal a significant sim-to-real deployment gap which primarily stems from unmodeled variations in motor parameters~\cite{miller2011parametrized} such as time-dependent torque limits.
To bridge this gap, we leverage a motor calibration procedure by utilizing rollouts from the real world. After the initial training in simulation with uncalibrated parameters, the policy is deployed on the real-world robot to collect extensive calibration data, including joint positions, velocities, commanded torques, and actual torque measurements. Using CMA-ES~\cite{nomura2024cmaes}, the torque limits are optimized as functions of joint state from collected real-world data. The policy is subsequently fine-tuned in simulation using these calibrated parameters.
% Remarkably, this calibration process requires only a single iteration to achieve reliable real-world performance, demonstrating both efficiency and effectiveness.
%% KF(04/19): The sentences below are hard to comprehend for people who are not familiar with the context. They also read like trivial implementation details. We would need to better explain why and how in concise words and highlight the key novelty here.
% We hypothesize that this discrepancy stems from inaccurate torque limits in the simulation. Specifically, when one leg is lifted, the remaining legs are more likely to operate near their torque limits.
% \subsection{Flexible Target Specification}
\subsection{Task Interface for Versatile Loco-Manipulation}
\label{subsec:task_interface}
Leveraging the flexibility of our formulation and the adaptability of the \algabrvname controller, we solve diverse and complex loco-manipulation tasks based on user commands.
We interface with user input at the task level through three modalities:
\begin{itemize}[leftmargin=10pt, itemsep=0pt, parsep=0pt, topsep=0pt]
\item \textbf{Direct targets:} The most straightforward approach is to manually specify target trajectories for designated limbs, such as through teleoperation. These trajectories can be directly converted into the task representation described in \Cref{subsec:overview} with minimal post-processing.
\item \textbf{Contact points.} Many loco-manipulation tasks can be described by specifying key contact points and associated motions. Given these, target trajectories are generated via motion planning algorithms.
\item \textbf{Language instructions.} Free-form language provides a flexible way to describe complex tasks. While our controller is not directly conditioned on language, contact points and trajectories can be inferred from RGBD observations using vision-language models (VLMs)~\citep{openai2024gpt4ocard}, which provide strong semantic reasoning capabilities.
\end{itemize}
These different modalities reflect varying levels of user specification and prior knowledge, resulting in different levels of difficulty for equivalent tasks expressed via different modalities. Additional implementation details for each interface modality are provided in the Appendix.
% Given the learned flexible command-level controller, we introduce three task-level interfaces to generate targets for solving diverse loco-manipulation tasks.
% We introduce a contact-based representation to enable specification of diverse loco-manipulation tasks. Denoted as $c = \{ c_\lambda \}_{\lambda \in \Lambda}$, the contact-based representation specifies the contact $c_\lambda$ between each end-effector $\lambda \in \Lambda$, either gripper or foot, and the environment in each subtask.
% Each $c_\lambda$ consists of the mask $m_\lambda$ as a boolean indicator of the contact, a contact point, a pre-contact point, and a post-contact point. The mask indicator decides if the limb is used for manipulation.
% If the indicator is positive, then the robot needs to move the designated end-effector to make contact with the environment by going through the pre-contact, contact, and post-contact points.
% The execution for each limb is synchronized with all designated end-effectors asked to reach the contact points at the same time.
% Given the contact-based targets $c_\lambda$, the target trajectories $\xi$ can be computed as introduced in \Cref{subsec:overview}.
% Such representations can come from either annotations by human users or prediction by a vision-language model (VLM) using the visual observation.
% For a complex task, we can decompose it into a series of $K$ subtasks, with each subtask $0 \leq k \leq K-1$ specified by a contact-based representation $c^k$.
% \textbf{Direct Targets.} Using a teleoperation interface, we directly send torso and end effector tracking targets via different input devices (see details in Appendix). This serves as an upper bound on the capability of the flexible low-level controller.
% \textbf{Contact points.} An interactive 3D point cloud user interface (UI) is developed to facilitate contact annotation. The users are prompted to select contact points $c$ from the point cloud, specify the contact mask $m$, and choose the corresponding pre-contact and post-contact directions. The UI re-prompts the user for contact selection after the subtask is completed. See details in Appendix.
% \textbf{Language instructions.} A VLM framework is designed to replace the contact annotation process. The raw 2D image observation is segmented by SAM2~\cite{ravi2024sam2}, and the masks and task information are provided to GPT-4o~\cite{achiam2023gpt} to select the task-related masks. Key points are then sampled in the masks, and GPT4o is prompted to select the contact points. The selected points in the image are mapped to point clouds to replace the contact annotations. See details in the Appendix.
\begin{figure}[t]
\centering
\includegraphics[width=\linewidth]{figures/gait.pdf}
\caption{\textbf{Flexible Gait Transitions.} With \algabrvname, the robot can execute a range of gaits, including four-legged trotting and three-legged bouncing with a designated limb lifted (FL: front-left, FR: front-right, HL: hind-left, HR: hind-right). The controller enables seamless transitions between these gait modes in real-world operation, without requiring the robot to pause or reset its stance.}
\label{fig:gait_switch}
\vspace{-1em}
\end{figure}
%% KF (4/26): These are moved here without adaptation yet.
%===============================================================================
\section{Experiments}
\label{sec:experiments}
We conduct a series of real-world experiments to investigate the following questions: \textbf{Q1.} Can the \algabrvname controller robustly perform interlimb coordination? (\Cref{sec:experiments_lowlevel})\ \textbf{Q2.} Can \algabrvname successfully perform diverse and complex loco-manipulation tasks on command? (\Cref{sec:experiments_highlevel})\ \textbf{Q3.} What are the primary sources of failure within the control stack? (\Cref{sec:experiments_failure_breakdown})
Experiments are run on a Boston~Dynamics \emph{Spot} (quadrupedal, 12 actuators) equipped with \emph{Spot Arm} and a gripper (7 actuators total). The onboard stereo cameras and IMU supply state estimates and RGB-D images for the task interface. All computation executes on an external PC connected over Ethernet. Additional hardware details appear in the Appendix.
% \subsection{Low-Level Controller Evaluation}
\subsection{Quantitative Analysis on Interlimb Coordination}
\label{sec:experiments_lowlevel}
We first conduct quantitative analysis on the \algabrvname controller's performance on interlimb coordination.
Specifically, we examine the robot's ability to (i) switch gaits on demand under varying arm and leg configurations and (ii) perform target-driven end-effector motions with multiple limbs.
\begin{wrapfigure}{r}{0.5\textwidth}
\centering
\vspace{-10pt}
\includegraphics[width=\linewidth]{figures/tracking_error.pdf}
\captionof{figure}{\textbf{End-Effector Motions.}
The robot tracks independent rhombus-shaped trajectories using the arm and the lifted front-left (FL) leg while walking with three supporting limbs.
\textbf{(A)} Overlay of reference and executed trajectories for both effectors.
\textbf{(B)} Linear tracking errors in $x$ and $y$ directions and rotational error in yaw.
% The low tracking error across both limbs demonstrates the controller’s ability to achieve precise interlimb coordination during dynamic locomotion.
%Arm and FL leg simultaneously tracking rhombus references.
}
\label{fig:tracking}
\vspace{-10pt}
\end{wrapfigure}
\textbf{Gait transitions.}
We command the robot to switch between four-legged trotting and three-legged bouncing, while randomly changing the end-effector targets during execution.
As shown in \Cref{fig:gait_switch}, the robot adapts its gait dynamically to maintain balance throughout the transitions. All gait switches occur instantly, without requiring the robot to stop or transition through a predefined mode. We observe that the robot can immediately reassign the lifted limb from FL to HR during walking without interruption.
In contrast, prior work~\citep{wolfslag2020optimisation,cheng2023legs,arm2024pedipulate,lin2024locoman} for leg manipulation rely on predefined state machines or pre-trained gait policies often requiring the robot to pause before switching limbs.
% We command the robot to
% switch between four-legged trotting and
% three-legged
% walking and randomly change the end-effector targets during the experiment.
% As shown in \Cref{fig:gait_switch}, the robot dynamically adapts its gait while maintaining balance. All the gait-switching happens instantly without the need to stand still or mode shifting (\eg, robot can directly switching the lift-up leg from FL to HR during walking). In comparison, all previous works involving using legs as manipulators require explicit state machine to change the mode, during which time the robot stand still.
\textbf{End-effector motions.} While walking with three support legs, the robot is asked to use the arm and the lifted FL leg to independently track rhombus-shaped trajectories ten times in succession.
\Cref{fig:tracking}(A) overlays the reference and executed paths, and \Cref{fig:tracking}(B) reports the linear tracking errors in $x$, $y$, and the rotational error for yaw. The low mean Cartesian error across both effectors confirms that the controller maintains precise interlimb coordination during dynamic locomotion.
\subsection{Loco-Manipulation Task Solving}
\label{sec:experiments_highlevel}
\begin{figure}
\centering
\includegraphics[width=\linewidth]{figures/result.pdf}
\caption{\textbf{Loco-Manipulation Task Performances.}
We evaluate \algabrvname on 12 real-world loco-manipulation tasks, reporting the success rate over 10 trials per task. Our three ReLIC variants consistently outperform the baselines, demonstrating the versatility and robustness of our controller. In contrast, the end-to-end RL (E2E) and MPC baselines fail on most tasks, highlighting the challenges of dynamic interlimb coordination and the advantage of our approach.
% We evaluate \algabrvname across 12 tasks, reporting the success rate over 10 trials for each.
% The baseline methods rarely succeed, highlighting both the difficulty of these tasks and the effectiveness of our hierarchical design.
}
\label{fig:result}
\end{figure}
\textbf{Tasks.} As shown in \Cref{fig:execution}, we devise 12 tasks in the real world to demonstrate the capability of \algabrvname and conduct comprehensive evaluation, examining the robot's capability involving diverse and complex interlimb coordination patterns (detailed task setup can be found in Appendix):
\begin{itemize}[leftmargin=1em, itemsep=0pt, topsep=0pt]
\item \textbf{Mobile interlimb coordination:} \textit{Yoga Ball} and \textit{Shipping Box} test the robot’s ability to manipulate large objects using its arm and one leg while navigating with the remaining three legs.
\item \textbf{Stationary interlimb coordination:} In the \textit{Tire Pump}, \textit{Trash Bin}, \textit{Deck Box}, and \textit{Small Bin} tasks, the robot coordinates its arm with one designated leg for object manipulation while maintaining balance through static support from the remaining three legs.
\item \textbf{Foot-assisted manipulation:} While the \textit{Tool Chest}, \textit{Storage Bin}, \textit{Chair}, \textit{Basket}, \textit{Drawer}, and \textit{Laundry Bag} tasks can be completed using only the arm, incorporating an additional leg as a manipulator demonstrates measurable performance improvements in stability and task execution.
\end{itemize}
\textbf{Model variants and baselines.}
We evaluate three variants of our method based on different user input modalities: direct targets (\textbf{ReLIC‑Direct}), contact points (\textbf{ReLIC‑Contact}), and language instructions (\textbf{ReLIC‑Language}). ReLIC‑Contact and ReLIC‑Language are evaluated on a subset of tasks due to input modality constraints.
All variants uses the same trained \algabrvname
controller.
As baselines, we compare against an end-to-end reinforcement learning policy~\citep{fu2023deep} (\textbf{E2E}) and a model predictive control (\textbf{MPC}) policy conditioned on direct targets.
% The end-to-end RL baselines failed in all tasks due to unstable sim2real performance and poor end-effector tracking. The MPC controller doesn't support inter-limb manipulation but is accurate and robust in end-effector control, as demonstrated in solving the \textit{Tire-Pump} task (see Appendix for experiment details).
\begin{figure}
\centering
\includegraphics[width=\linewidth]{figures/interface.pdf}
\caption{\textbf{Flexible Interlimb Coordination.} \algabrvname enables dynamic assignments of limbs between manipulation and locomotion during task execution. This seamless role-switching allows the robot to adapt efficiently to varying task demands. Here, we demonstrate the different interlimb coordination patterns during the execution of \textit{Shipping Box} \textbf{(A)}, \textit{Deck Box} \textbf{(B)}, and \textit{Chair} \textbf{(C)} tasks. }
\label{fig:interface}
\end{figure}
\textbf{Results.}
We evaluate \algabrvname on the 12 loco-manipulation tasks in unstructured real-world environments. Each model variant and baseline is evaluated over 10 randomized trials per task. The average success rate for each settings is reported in \Cref{fig:result}.
\algabrvname-Direct achieves the highest success rates across all but one task. In 9 out of 12 tasks, it succeeds in more than 8 out of 10 trials, covering all three task categories. Most tasks involve long-horizon execution with multiple stages, requiring diverse capabilities such as object picking (\eg, \textit{Basket}, \textit{Small Bin}), displacement (\eg, \textit{Storage Bin}, \textit{Drawer}), locomotion (\eg, \textit{Yoga Ball}, \textit{Laundry Bag}), and maintaining forceful contact (\eg, \textit{Trash Bin}, \textit{Chair}). The consistently high performance demonstrates the robustness and reliability of the \algabrvname controller.
On tasks with additional challenges of repetitive motion or fine balance, like \textit{Deck Box} and \textit{Tire Pump}, \algabrvname-Direct maintains a non-negligible success rate, though performance is reduced. Among all tasks, \textit{Shipping Box} proves the most difficult due to the large size and rigid-body dynamics of the object.
Despite receiving less direct guidance, both \algabrvname-Contact and \algabrvname-Language achieve comparable results. In these settings, the controller reliably executes targets inferred user-specified contact points or language instructions, validating the effectiveness of our hierarchical framework.
Baseline methods underperform across the board due to their lack of robust interlimb coordination. The off-the-shelf MPC baseline lacks support for interlimb manipulation or three-leg locomotion.
As a result, it fails on all tasks except for \textit{Tire Pump}.
This success occurs only by chance when the robot happens to step on the pump and actuate it with its arm, without coordinated intent.
The end-to-end RL baseline fails across all tasks due to unstable gait generation and inaccurate end-effector tracking, underscoring the need for more structured and adaptive control strategies.
%
% \algabrvname-Direct performs relatively poorly on the \textit{Deck Box} and \textit{Tire Pump} tasks. In \textit{Deck Box}, the robot must lift the lid using one leg while simultaneously placing a cloth inside using the arm. This requires the leg to be raised to a relatively high position and maintain forceful contact with the lid to keep it open, which significantly challenges the robot’s balance. In \textit{Tire Pump}, the robot needs to stabilize the pump with its arm while using a leg to push it. This task demands high precision from the controller, as the repetitive pushing motion introduces cyclical disturbances that make it difficult for the robot to maintain stability.
%
% % \textit{Shipping Box} is deceptively the most difficult task. \algabrvname-Direct struggles to complete it primarily due to the box’s large size and rigid-body nature. Even slight deviations from the optimal grasping point can cause the box to tilt, making stable manipulation difficult. The combination of its size and the need for coordinated interlimb mobility makes this task particularly challenging.
% which enables smooth execution once suitable targets are provided. Crucially, whenever the ``correct'' targets are
% selected, the controller executes them with tight tracking across pure-arm, pure-leg, and interlimb modes (\Cref{fig:interface}).
% Although provided with less direct human specifications, \algabrvname-Contact and \algabrvname-Language still achieve comparable results on the evaluated tasks. For \algabrvname-Contact, the controller is usually able to achieve targets computed from user-specified contact points.
% User can also provide meaningful corrections to the next target point given the outcome of previous robot motion, guaranteeing the success rate.
% As compared, \algabrvname-Language runs in a more "open-loop" manner where the full set of targets are specified at the begining. Therefore, it can only accomplish some relatively short-horizon tasks (i.e., \textit{Yoga Ball} and \textit{Tool Chest}), which does not requires a long sequence of targets. The targets provided by VLM are not always correct, this is the main cause of the performance gap of this method as compared to \algabrvname-Contact.
% Direct teleoperation performs best among all interfaces and serves as an upper-bound oracle. The contact-based interface allows users to effectively decompose tasks into contact-oriented stages (e.g., lifting a leg before dragging a laundry bag) and applies commonsense reasoning (e.g., stepping on a pedal to open a trash bin), enabling coordinated limb behaviors in both mobile (e.g., holding a yoga ball while walking) and stationary (e.g., closing a tool chest after placing an object inside) settings. The VLM-based interface further reduces human effort by automatically selecting contact points from RGB images using task descriptions and user-designed task-specific prompts.
% Despite strong overall performance, we observe that contact-based decomposition paired with trajectory planning struggles with contact-rich tasks requiring real-time physical feedback (\eg, \textit{Deck Box} and \textit{Shipping Box}). In these cases, direct teleoperation confirms that the low-level controller is capable of solving the task when supplied with live, adaptive targets—suggesting opportunities for future work on feedback-driven planning or imitation-based refinement.
% All variants benefit from the same flexible low-level
% controller, which enables smooth execution once suitable targets are provided. Crucially, whenever the ``correct'' targets are
% selected, the controller executes them with tight tracking across pure-arm, pure-leg, and interlimb modes (\Cref{fig:interface}).
% Different interfaces involve different amounts of human effort and prior knowledge. Direct teleoperation performs best among all interfaces and serves as an upper-bound oracle. The contact-based interface allows users to effectively decompose tasks into contact-oriented stages (e.g., lifting a leg before dragging a laundry bag) and applies commonsense reasoning (e.g., stepping on a pedal to open a trash bin), enabling coordinated limb behaviors in both mobile (e.g., holding a yoga ball while walking) and stationary (e.g., closing a tool chest after placing an object inside) settings. The VLM-based interface further reduces human effort by automatically selecting contact points from RGB images using task descriptions and user-designed task-specific prompts.
% Despite these promising results, we observe that contact-oriented task decomposition with trajectory planning cannot handle contact-rich tasks that require intricate physical interactions with real-time feedback (\eg, \textit{Deck Box} and \textit{Shipping Box}). In these cases, direct teleoperation—our upper-bound oracle—confirms that the low-level control is capable of solving the task if provided with real-time updated targets from humans, indicating potential future gains from more advanced planning algorithms or data-driven imitation policies.
% The baseline methods demonstrate inferior performance compared to our approach, primarily due to their inability to handle the robust interlimb coordination required by our task suite. The Boston Dynamics MPC stack, lacking dedicated interlimb coordination capabilities, fails on all tasks except the \textit{Tire Pump} scenario. In this singular case, it achieves a moderate success rate through trial-and-error positioning where the robot occasionally steps on the pump and subsequently operates the handle with its arm - a success enabled solely by the system's high-precision whole-body control.
% The end-to-end RL baseline performs poorly, failing across all tasks. This stems from limitations in tracking performance, suggesting that task-specific policy fine-tuning would be necessary to achieve meaningful success rate.
% In comparison, the end-to-end RL policy can hardly accomplish any of these tasks, due to the inaccurate tracking performance (see Appendix). MPC method cannot achieve any tasks that requires interlimb coordination since the original controller doesn't provide such interface. There is one special case, however, that in \textit{Tire Pump} MPC controller can step on the tire pump in normal walking gait with some trial-and-error, and then use the manipulator to push the pump, in this case it can achieve pretty high success rate given high-precision motion generated by the model-based control.
\begin{wrapfigure}{r}{0.45\textwidth}
\vspace{-10pt}
\begin{center}
\includegraphics[width=\linewidth]{figures/failure_breakdown.png}
\end{center}
\captionof{figure}{\textbf{Failure Breakdown.}
Analysis of failure modes for the \algabrvname-Contact variant across multiple tasks.
Failures are categorized into SLAM errors, tracking errors, balance loss, and inaccurate contact.}
\label{fig:error}
\vspace{-10pt}
\end{wrapfigure}
\subsection{Failure Analysis}
\label{sec:experiments_failure_breakdown}
With \algabrvname-Contact as a test case, we analyze failure modes with a breakdown illustrated in \Cref{fig:error}.
The first source of failure for \algabrvname-Contact comes from perception error, primarily due to inaccuracies in the 3D point cloud or state estimation.
Even when perception is accurate, task failures may still arise for tracking performed by the \algabrvname controller, particularly under extreme or unstable body configurations. For example, when the robot attempts to reach an arm target while one foot is stepping on the trash bin pedal, the unpredictable external force from the pedal poses significant challenges for the policy to maintain balance.
The third category of failures involves unintended manipulation errors with challenging contacts.
A common example for this category occurred in the \textit{Yoga Ball} task, where the robot either lost grip or unintentionally kicked the ball, resulting in slippage.
Such failures are especially challenging to avoid or recover from, as a viable solution would require real-time feedback and fine-grained contact reasoning. Despite these challenges, \algabrvname achieves an overall success rate of 82.5\% across the four most contact-intensive loco-manipulation tasks.
% ==============================================
% sankeymatic.com
% ==============================================
% Total [38] Perception Success
% Total [2] SLAM Inaccuracy
% Perception Success [35] Controller Success
% Perception Success [2] Tracking Failure
% Perception Success [1] Balancing Failure
% Controller Success [33] Task Success
% Controller Success [*] Inaccurate Contact
% :Total #ff7f0e
% :Perception Success #ffbf50
% :Controller Success #a2b627
% :Task Success #2ca02c
% :SLAM Inaccuracy #E15759
% :Tracking Failure #E15759
% :Balancing Failure #E15759
% :Inaccurate Contact #E15759
% ==============================================
%===============================================================================
\vspace{15px}
\section{Conclusion and Discussion}
\label{sec:conclusion}
We introduced \algabrvname, an approach that enables flexible interlimb coordination for versatile loco-manipulation.
By dynamically assigning limb roles and decoupling locomotion and manipulation into two coordinated modules, \algabrvname learns to achieve robust whole-body control.
Based on task specifications of various modalities, the \algabrvname controller can be employed to solve diverse and complex tasks on command.
Our system demonstrates strong real-world performance on 12 challenging scenarios, spanning mobile interlimb coordination, stationary interlimb coordination, and foot-assisted manipulation.
We hope this work inspires further research at the intersection of reinforcement learning and whole-body control, and highlights the critical role of interlimb coordination in advancing loco-manipulation capabilities.
% We present \algabrvname, a hierarchical framework that unifies contact-level reasoning and low-level adaptive control for whole-body loco-manipulation on legged robots. Unlike prior approaches that rely on fixed limb roles or handcrafted controllers, \algabrvname enables dynamic, limb-agnostic coordination across locomotion and manipulation tasks. Through contact-oriented task decomposition and a reinforcement learning-based control policy, the system supports seamless transitions between locomotion gaits and manipulation modes without requiring explicit state machines. Extensive real-world experiments on an arm-equipped quadruped robot demonstrate robust performance across a wide range of complex scenarios. These results highlight \algabrvname's ability to generalize across diverse contact configurations, advancing the frontier of adaptive, multi-contact robot behaviors in unstructured environments.
%===============================================================================
\newpage
\section{Limitations}
\label{sec:limitations}
While \algabrvname demonstrates versatile whole-body control across a diverse set of loco-manipulation tasks, several limitations remain.
First, the high-level task interface based on contact points and language instructions generates targets in an open-loop fashion. Although effective in the evaluated scenarios, this approach can be brittle in tasks requiring real-time adaptation or fine-grained feedback. A promising direction is to collect large-scale demonstrations and train policies to predict task-level actions through imitation learning.
Second, the manipulation controller currently relies on a standard inverse kinematics solver. For more complex tasks involving dynamic behaviors or collision avoidance, more expressive or learned controllers may be needed. Integrating such capabilities with the RL locomotion policy presents an interesting challenge for future work.
Finally, our experiments in this paper focus on repurposing legs for manipulation in an arm-mounted quadruped. Extending \algabrvname to broader forms of interlimb coordination, such as using arms to support locomotion or generalizing to other robot morphologies, offers exciting opportunities for expanding whole-body autonomy.
% While \algabrvname demonstrates robust whole-body control capabilities across a variety of loco-manipulation tasks, several limitations remain.
% The contact points selection pipeline benefits from the generalizability of human knowledge or foundation models; however, the sequences of end-effector targets generated from contact points are open-loop trajectories. In many contact-rich manipulations, this does not provide enough stability. To achieve similar or even better performance compared to direct teleoperation, either a more precise interface that allows users to select targets more accurately or a closed-loop reactive policy that can recover from failures is required. An effective way to solve this would be to collect large-scale demonstrations via teleoperation and use task-level targets as the action space to train imitation learning policies.
% Another current limitation is the lack of compliant manipulation controllers in the framework. Current end-effector tracking is whole-body inverse kinematics to calculate target joint positions. While this is simple and effective for free-space tracking, for force-mediated manipulation tasks like \textit{Shipping Box}, the end-effectors must avoid hard contact with the rigid body surface, and position tracking controllers are insufficient. The decomposition between manipulation and locomotion joints enables us to introduce more advanced variable impedance controllers to the end effector without compromising locomotion stability. In cluttered environments, it's also possible to add collision avoidance functionalities into the end-effector target planning. For tasks that teleoperation has a low success rate or is unable to solve, we can improve the controller as well as the teleoperation interface to solve more challenging tasks.
% Lastly, while the current locomotion policies are trained via reinforcement learning (RL) in the simulator, other parts of the method do not explicitly benefit from the physical simulator. We can use the simulator also in a decomposed setup, where we treat the always stable locomotion policy as a floating base and train multi-task manipulation policies for generalized environment and task distributions.
%===============================================================================
\clearpage
% The acknowledgments are automatically included only in the final and preprint versions of the paper.
% \acknowledgments{If a paper is accepted, the final camera-ready version will (and probably should) include acknowledgments. All acknowledgments go at the end of the paper, including thanks to reviewers who gave useful comments, to colleagues who contributed to the ideas, and to funding agencies and corporate sponsors that provided financial support.}
%===============================================================================
% no \bibliographystyle is required, since the corl style is automatically used.
\bibliography{example} % .bib
\input{appendix}
\end{document}
%!TEX root = root.tex
\usepackage{booktabs}
\PassOptionsToPackage{numbers,sort&compress}{natbib}
\usepackage{amsmath,amssymb,stmaryrd,mathtools}
\usepackage{xspace}
\usepackage{bbm}
\usepackage{mathtools}
\usepackage{bm}
\usepackage{graphicx}
\usepackage{lipsum}
\usepackage{wrapfig}
\usepackage{caption}
\usepackage{multicol, multirow}
\usepackage[table,dvipsnames]{xcolor}
\usepackage{siunitx}
\usepackage{enumitem}
\usepackage{authblk}
\usepackage{hyperref}
\usepackage{cleveref}
\usepackage{booktabs}
\usepackage[most]{tcolorbox}
\tcbset{
colback=gray!5, % background color
colframe=black, % border color
fonttitle=\bfseries,
boxrule=0.5pt,
arc=2mm,
left=1mm,
right=1mm,
top=1mm,
bottom=1mm,
}
\setlipsum{%
par-before = \begingroup\color{gray},
par-after = \endgroup
}
\definecolor{orange(sae/ece)}{rgb}{1.0, 0.49, 0.0}
\definecolor{teal(sae/ece)}{rgb}{0, 0.47, 0.52}
\definecolor{purple}{rgb}{0.74, 0.65, 1.0}
\definecolor{light_gray}{rgb}{0.9, 0.9, 0.9}
\definecolor{medium_gray}{rgb}{0.6, 0.6, 0.6}
\definecolor{dark_gray}{rgb}{0.2, 0.2, 0.2}
\definecolor{dark_blue}{rgb}{0.098, 0.239, 0.52}
\definecolor{dark_brown}{rgb}{0.3255, 0.004, 0.001}
\definecolor{dark_purple}{rgb}{0.478, 0.1569, 0.4863}
\definecolor{light_blue}{rgb}{0.33, 0.80, 1}
\definecolor{relic_red}{rgb}{0.87,0.36,0.42}
\definecolor{relic_green}{rgb}{0.47,0.82,0.34}
\definecolor{relic_purple}{rgb}{0.55,0.40,0.81}
% \newcommand{\algname}{Flexible Interface for Loco-Manipulation\xspace}
% \newcommand{\algabrvname}{FILM\xspace}
% \newcommand{\algname}{Contact-Adaptive Loco-Manipulation\xspace}
% \newcommand{\algabrvname}{CALM\xspace}
\newcommand{\algname}{Reinforcement Learning for Interlimb Coordination\xspace}
\newcommand{\algabrvname}{ReLIC\xspace}
\newcommand{\filler}[1]{ {\color{gray}\lipsum[1-#1]}}
\newcommand{\xz}[1]{{\color{red}XZ: #1}}
\newcommand{\kf}[1]{{\color{orange}KF: #1}}
\newcommand{\yc}[1]{{\color{violet}YC: #1}}
\newcommand{\ls}[1]{{\color{teal}LS: #1}}
\newcommand{\notes}[1]{{\color{black} #1}}
\newcommand{\deprecated}[1]{{\color{blue}======== Deprecated ========\\ #1\\================}}
% Omit final dot from each def.
\usepackage{expl3}
\ExplSyntaxOn
\newcommand\latinabbrev[1]{
\peek_meaning:NTF . {% Same as \@ifnextchar
#1\@}%
{ \peek_catcode:NTF a {% Check whether next char has same catcode as \'a, i.e., is a letter
#1.\@ }%
{#1.\@}}}
\ExplSyntaxOff
\def\eg{\latinabbrev{e.g}}
\def\etal{\latinabbrev{et al}}
\def\etc{\latinabbrev{etc}}
\def\ie{\latinabbrev{i.e}}
\newtheorem{definition}{Definition}
\newcommand{\daoaf}{\textcolor{orange}{\textbf{DPA-OMF}}\xspace}
\newcommand{\daadef}{\textcolor{purple}{\textbf{DPA-ADEF}}\xspace}
\newcommand{\tcc}{\textcolor{gray}{\textbf{TCC}}\xspace}
\newcommand{\tccot}{\textcolor{dark_gray}{\textbf{TCC-OT}}\xspace}
\newcommand{\datf}{\textcolor{dark_blue}{\textbf{DPA-TF}}\xspace}
\newcommand{\SFTOA}{\textcolor{light_blue}{\textbf{SFT-bestOA}}\xspace}
% abbrev.
\newcommand{\mvpotfinetuneabbrev}{\textcolor{light_blue}{\textbf{FT-MVP-OT}}\xspace}
\newcommand{\imagenetabbrev}{\textcolor{dark_brown}{\textbf{ImNet-OT}}\xspace}
\newcommand{\gt}{\textcolor{teal}{\textbf{GT}}\xspace}
\newcommand{\imagenet}{\textcolor{dark_brown}{\textbf{ImageNet-OT}}\xspace}
\newcommand{\SFTade}{\textcolor{dark_purple}{\textbf{SFT-bestADE}}\xspace}
% Commands for mathematical notations
% Representation notation
\newcommand{\robot}{\mathrm{R}}
\newcommand{\human}{\mathrm{H}}
\newcommand{\repr}{\phi_\robot}
\newcommand{\reph}{\phi_\human}
\newcommand{\approxreph}{\tilde{\phi}_\human}
\newcommand{\rewr}{r} % robot reward
\newcommand{\rewh}{r^*} % robot reward
\newcommand{\obs}{o}
\newcommand{\obstraj}{\mathbf{o}}
\newcommand{\state}{s}
\newcommand{\policyr}{\pi_\robot} % robot policy
\newcommand{\policyh}{\pi_\human} % human policy
% Human notation
\newcommand{\hdyn}{f_\mathrm{H}}
\newcommand{\xH}{x_\mathrm{H}}
\newcommand{\hderiv}{\dot{x}_\mathrm{H}}
\newcommand{\uH}{u_\human}
\newcommand{\uHopt}{u^*_\human}
\newcommand{\hxdim}{{n_\mathrm{H}}}
\newcommand{\hudim}{{m_\mathrm{H}}}
% Time notation
\newcommand{\tvar}{t}
\newcommand{\thor}{T}
\newcommand{\dt}{\Delta t}
% Parameter notation
\newcommand{\param}{\theta} % unknown parameter (dynamics or objective)
\newcommand{\weights}{\phi} % weights of NN
\newcommand{\paramest}{\hat{\theta}}
\newcommand{\belief}{b}
\newcommand{\paramdyn}{f_L}
\newcommand{\approxparamdyn}{f^{\weights}_L}
% Joint human and parameter notation
\newcommand{\jointstate}{x}
\newcommand{\jointdyn}{f}
% Relative state and dynamics.
\newcommand{\xrel}{x_\mathrm{rel}}
\newcommand{\reldyn}{f_\mathrm{rel}}
\newcommand{\rel}{\mathrm{rel}}
% Robot notation
\newcommand{\rdyn}{f_\mathrm{R}}
\newcommand{\xR}{x_\mathrm{R}}
\newcommand{\rderiv}{\dot{x}_\mathrm{R}}
\newcommand{\uR}{u_\mathrm{R}}
\newcommand{\uRopt}{u^*_\mathrm{R}}
\newcommand{\rxdim}{n_\mathrm{R}}
\newcommand{\rudim}{{m_\mathrm{R}}}
\newcommand{\rgoal}{{g_\mathrm{R}}}
% Trajectory notation.
\newcommand{\jointstatetraj}{\mathbf{\jointstate}}
\newcommand{\uRtraj}{\mathbf{\uR}}%{\uR^{0:\tvar}}
\newcommand{\uHtraj}{\mathbf{\uH}}%{\uH^{0:\tvar}}
\newcommand{\obsuRtraj}{\mathbf{\hat{u}_\mathrm{R}}}%{\uR^{0:\tvar}}
\newcommand{\obsuHtraj}{\mathbf{\hat{u}_\mathrm{H}}}%{\uH^{0:\tvar}}
\newcommand{\uHtrajopt}{\mathbf{\uHopt}}
\newcommand{\uRtrajopt}{\mathbf{\uRopt}}
\newcommand{\tildeuRtraj}{\mathbf{\tilde{u}_\mathrm{R}}}
\newcommand{\tildeuHtraj}{\mathbf{\tilde{u}_\mathrm{H}}}
% Set of likely human trajectories over time horizon [0,t]
\newcommand{\uHtrajset}{\mathbb{U}_\mathrm{H}}%{\mathbb{U}^{0:\tvar}_\mathrm{H}}
\newcommand{\unlikelyuHtrajset}{\mathbb{S}_\mathrm{H}}%{\mathbb{U}^{0:\tvar}_\mathrm{H}}
% For reachability stuff
\newcommand{\R}{\mathbb{R}}
\newcommand{\uRset}{\mathcal{U}_\mathrm{R}}
\newcommand{\uHset}{\mathcal{U}_\mathrm{H}}
\newcommand{\uHsetapprox}{\widetilde{\mathcal{U}}_\mathrm{H}}
\newcommand{\targetfun}{\ell}
\newcommand{\targetset}{\mathcal{L}}
\newcommand{\vfun}{V}
\newcommand{\brs}{\mathcal{V}} % full BR
\newcommand{\approxbrs}{\widetilde{\mathcal{V}}} % Bayesian BRT
\newcommand{\truebrs}{\mathcal{V}^*}
% For proof utilities
% \newtheorem{remark}{Remark}
% \newtheorem{definition}{Definition}
%\newtheorem{theorem}{Theorem}
%\newtheorem{proposition}{Proposition}
%\newtheorem{lemma}{Lemma}
%\newtheorem{corollary}{Corollary}
\usepackage{amsmath}
\DeclareMathOperator*{\argmax}{arg\,max}
\DeclareMathOperator*{\argmin}{arg\,min}
\clearpage
\appendix
% \begin{center}
% {\LARGE\bfseries Supplementary Materials of \\
% Versatile Loco-Manipulation through \\
% Flexible Interlimb Coordination
% } \\[1.5em]
% Paper ID 925 \\[1.5em]
% \href{https://relic-locoman.github.io/}{\textcolor{magenta}{https://relic-locoman.github.io/}} \\
% \end{center}
\numberwithin{equation}{section}
% \section{Website}
% Check \url{https://relic-locoman.github.io/} for videos showing all interlimb coordination loco-manipulation tasks and ablation studies.
\section{Notations}
\begin{table}[h]
\centering
\caption{Definition of symbols.}
\label{tab:notations}
\begin{tabular}{p{0.25\linewidth} p{0.65\linewidth}}
\toprule
\textbf{Symbol} & \textbf{Description} \\
\midrule
$\bm{v}_B$ & Linear velocity of the base in robot frame \\
$\bm{\omega}_B$ & Angular velocity of the base in robot frame \\
$\bm g$ & Gravity direction of the robot's base \\
$\bm q$ & Joint positions of the robot \\
$\bm \tau$ & Applied joint torque of the robot \\
$\dot{\bm q}$ & Joint velocities of the robot \\
$\bm{a}$ & Network's prediction \\
$\bm{x}_{rot}$ & Robot attitude (roll, pitch, and height) \\
\midrule
$\bm{v}_B^*$ & Target linear velocity of the base in robot frame \\
$\bm{\omega}_B^*$ & Target angular velocity of the base in robot frame \\
$\bm q^*$ & Target joint positions of the robot \\
$\bm{x}_{rot}^*$ & Target robot attitude \\
\midrule
$\mathbbm{1}_{walking}$ & True \textbf{iff} target velocities are non-zero. \\
$\mathbbm{1}_{flight}$ & True \textbf{iff} fewer than three feet are in contact with the ground. \\
$t_c^j$ & Contact time of foot $j$ \\
$t_a^j$ & Air time of foot $j$ \\
$T_{gait}$ & Gait cycle time \\
\midrule
$\prescript{w}{}{P}^{(i)}$ & Point cloud observation \\
$\prescript{w}{}{x}^{(i)}_{arm}$ & Desired 3D location of the arm end-effector \\
$\prescript{w}{}{R}^{(i)}_{arm}$ & Desired 3D rotation of the arm end-effector \\
$\prescript{w}{}{x}^{(i)}_{leg}$ & Desired 3D location of the leg end-effector \\
$\prescript{w}{}{R}^{(i)}_{leg}$ & Desired 3D rotation of the leg end-effector \\
$\prescript{w}{}{p}^{(i)}_{body}$ & Desired body pose \\
$m^{(i)}$ & Desired leg mask for manipulation or locomotion use \\
$g^{(i)}$ & Gripper state \\
$\prescript{w}{}{\xi}^{(i)}_\text{arm}$ & Arm end-effector trajectory \\
$\prescript{w}{}{\xi}^{(i)}_\text{leg}$ & Leg end-effector trajectory \\
$\prescript{w}{}{\xi}^{(i)}_\text{torso}$ & Torso end-effector trajectory \\
% \midrule
% $_A\bm r_{AB} \in \mathbb{R}^3$ & Position of frame B from frame A in frame A \\
% $\bm \Phi_{AB} \in SO(3)$ & Orientation of frame B in frame A \\
% $_A\bm v_{AB} \in \mathbb{R}^3$ & Linear velocity of frame B from frame A in frame A \\
% $_A\bm \omega_{AB} \in \mathbb{R}^3$ & Angular velocity of frame B from frame A in frame A \\
% $\boxminus : SO(3) \times SO(3) \rightarrow \mathbb{R}^{3}$ & Difference between two orientations as a rotation error vector \cite{bloesch2017iterated} \\
% \midrule
% $B$ & Frame attached to the base of the robot \\
% $E$ & Frame attached to the arm's end-effector of the robot \\
% $H$ & Frame attached to the handle's center on the object \\
% $I$ & Inertial frame (fixed to the initial frame $H$ on execution) \\
% \midrule
% $\bm h_\textrm{com}$ & Centroidal momentum of the robot \\
% $\bm q_b = (_I\bm r _{IB}, \bm \Phi_{IB})$ & Base pose of the robot \\
% $\bm q_o$ & Joint positions of the object \\
% $\dot{\bm q}_j$ & Joint velocities of the robot \\
% $\dot{\bm q}_o$ & Joint velocities of the object \\
% $\ddot{\bm q}_j$ & Joint acceleration of the robot \\
% $\bm\tau_j$ & Joint torques applied to the robot \\
% \midrule
% $\bm m$ & Manipulation contact mode between robot and the object \\
% $\bm x_r = (\bm h_\textrm{com}, _I\bm r _{IB}, \bm \Phi_{IB}, \bm q_j)$ & Kinematic state of the robot \\
% $\bm x_o = (\bm q_o, \dot{\bm q_o})$ & Kinematic state of the object \\
% $\bm x = (\bm x_r, \bm x_o)$ & Kinematic state of the robot and the object \\
% $\bm M= \{\bm m_t\}_{t=1}^{T_{\textrm{task}}}$ & Manipulation schedule (Sequence of manipulation contact modes) \\
% $\bm X= \{\bm x_t\}_{t=1}^{T_{\textrm{task}}}$ & Kinematic state trajectory (Sequence of kinematic states) \\
% $T_{\textrm{task}}$ & Duration (in s) of the trajectory for the task \\
% ${\mathbbm{1}_{object}:= \mathbbm{1}_{object}\left(\bm m\right)}$ & True \textbf{iff} at least one end-effector is either already in contact with or is establishing contact with the object\\
% $\mathbbm{1}_{prehensile}:= \mathbbm{1}_{prehensile}\left(\bm m\right)$ & True \textbf{iff} a prehensile interaction is active \\
% \midrule
% $\phi \in [0, 1] $ & Phase signal that helps index the reference trajectory $(\bm X^*, \bm M^*)$ \\
% $v_t^\phi$ & Task phase rate (first order-dynamics model for $\phi$) \\
% $\hat{v}_t^\phi$ & Task phase rate based on reward-dependent functions \\
% $\bar{\phi}$ & Nominal phase signal computed using $v^\phi = \frac{1}{T_{\textrm{task}}}$ \\
% $\delta_v$ & Residual phase rate (output from the policy)\\
% \midrule
% $dt$ & Step-size (in s) of the environment \\
% $\bm o_t$ & Observation from the environment at time-step $t$ \\
% $\bm a_t$ & Action applied to the environment at time-step $t$ \\
% $r_t$ & Reward from the environment at time-step $t$ \\
\bottomrule
\end{tabular}
\end{table}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{Hardware Setup}
\label{app:hardware}
The arm-mounted quadruped robot we use in this project is Boston Dynamics' Spot with a Spot Arm mounted on its back~\citep{bostondynamics_spot}. There are 12 actuators for the Spot, and each leg has three. The arm has six degrees of freedom and has a gripper at the end effector. The system is shown in \Cref{fig:app:spot_and_arm}.
In command-level control, we query the Spot API~\citep{bostondynamics_api} for the real-time Spot joint state estimates. For the reinforcement learning locomotion policy, we input the proprioceptive states concatenated with the torso targets into the neural network trained by RL, represented in ONNX Runtime~\citep{onnxruntime}. The computed joint targets from RL policy and IK planners are masked and combined together as a 19-dimension target joint vector for all Spot leg and arm joints and sent to the robot using the low-level motor control API~\citep{bostondynamics_api}.
\begin{figure}[h]
\centering
\includegraphics[width=0.5\linewidth]{figures/spot.png}
\caption{
% Boston Dynamics Spot and the mounted Spot Arm on the back.
\textbf{Robot Hardware.} Boston Dynamics Spot quadruped robot with the Spot Arm mounted on its back. This platform is used for all loco-manipulation experiments presented in this work.
}
\label{fig:app:spot_and_arm}
\end{figure}
%% KF(05/07): Need to briefly talk about how we get the point cloud. Perhaps we can provide a bit more details about Spot API/SDK and the low-level control interface of the Spot. Briefly mention that the Spot can also be controlled by the SDK?
For task specifications on the task level, we use the built-in stereo cameras on Spot to get RGBD images and construct the colored point clouds used in the \textit{contact point} and \textit{language instruction} based interfaces.
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% \section{Task Design}
% % \label{app:exp_locomanipulation_task}
% \label{app:task}
% In this section, we discuss the experiment details for the reported loco-manipulation tasks in \Cref{sec:experiments_highlevel}.
% %
% % \subsection{Interlimb Coordination Task Descriptions}
% % \label{app:task}
% %
% We define 12 real-world tasks to demonstrate the capability of \algabrvname as shown in \Cref{fig:execution}, divided into three categories.
% Some of the tasks have slightly different variants tailored for different interfaces. We use \textbf{\textit{D}}, \textbf{\textit{C}}, and \textbf{\textit{L}} to denote the interface using \textit{Direct targets}, \textit{Contact points}, and \textit{Language instructions}, respectively. Please refer to \Cref{app:direct_targets}, \Cref{app:contact_points}, \Cref{app:language_intstructions} for further details.
% \subsection{Mobile Interlimb coordination}
% This category includes two challenging tasks demonding the robot to manipulate large objects using its arm and one leg while navigating with the remaining three legs.
% \begin{itemize}
% % \item \textit{Yoga Ball-\textbf{D}:} A 55cm yoga ball is placed on the base. The robot is required to pick up the ball and place it into a nearby laundry basket.
% % \item \textit{Yoga Ball-\textbf{C}/Yoga Ball-\textbf{L}:} A 55cm yoga ball is placed on the base. The robot is required to pick up the ball and move backwards.
% \item \textit{Yoga Ball:} A $55cm$ yoga ball is placed on the base. The robot is asked to: pick up the ball and place it into a nearby laundry basket (\textbf{D}); pick up the ball and move backward (\textbf{C, L}).
% \item \textit{Shipping Box}: A large cardboard shipping box is randomly placed. The robot is requested to pick up the box and place it onto a pre-defined destination (\textbf{D}).
% \end{itemize}
% \textbf{Stationary interlimb coordination:} This category includes four tasks that require the robot to coordinate its arm with one designated leg for object mnipulation while maintaining balance through static support from the remaining three legs.
% \begin{itemize}
% \item \textit{Trash Bin} An eight-gallon step-on trash bin is placed in front of the robot. The robot is asked to: pick up the Coke can on the ground, step on the trash bin pedal with one leg to open the lid, and throw the can into the bin (\textbf{D}); with the Coke can in the gripper, step on the trash bin pedal with one leg to open the lid, and throw the can into the bin (\textbf{C}).
% \item \textit{Deck Box:} A large deck box is placed in front of the robot. A folded blanket is placed between the robot and the deck box on the ground. The robot is asked to grasp the blanket with the arm and use one leg to open the deck box lid and keep it open while using the arm to put the blanket into the deck box (\textbf{D}).
% \item \textit{Tire Pump:} A small tire pump is connected to a flat bike tire. The robot is asked to stabilize the tire pump with the arm and use one leg to step on the pump to inflate the tire (\textbf{D}).
% \item \textit{Small Bin:} A one-gallon bathroom step-on trash bin is placed randomly. The robot is asked to grasp a paper ball from the floor, move to the trash bin, step on the trash bin pedal with one leg to open the lid, and throw the paper into the bin (\textbf{D}).
% \end{itemize}
% \textbf{Foot-assisted manipulation:} This category includes six tasks that can be completed using only the arm. However, incorporating an additional leg as a manipulator can improve the task performance by either
% \begin{itemize}
% \item \textit{Tool Chest:} A big tool chest has one or two open drawers. The robot is asked to: pick up a blue tape on the ground, place it into the drawer, and then close it with one leg (\textbf{D}); with a blue tape holding in the end-effector, place the tape into the drawer and then close the drawer with one leg (\textbf{C}); close top and down two drawers simultaneously with the arm and a leg (\textbf{L}).
% % \item \textit{Tool Chest-\textbf{C}:} A big tool chest has one drawer wide open. The robot has a blue tape holding in the end-effector. It needs to place the tape into the drawer and then close the drawer with one leg.
% % \item \textit{Tool Chest-\textbf{L}:} A big tool chest has two drawers wide open, one on the top and the other on the bottom. The robot needs to close both drawers simultaneously with the arm and a leg.
% \item \textit{Basket:} A shopping basket is placed on the floor. The robot is asked to: grasp a paper towel from the shelf in front of the robot, put it into the basket, and use one leg to carry the basket while reaching for the paper towel (\textbf{D}); with a toy held in the end-effector, place the toy into the basket, and then use one leg to lift the basket and move backward (\textbf{C}).
% % \item \textit{Basket-\textbf{C}:} The robot has a toy held in the end-effector. The robot should first place the toy into the basket, and then use one leg to lift the basket and move backwards.
% \item \textit{Drawer:} Several toys are randomly placed on the ground. The robot is requested to pick up the toys one by one and put them on a small nightstand. The robot needs to open and close the nightstand's top drawer with one leg (\textbf{D}).
% \item \textit{Storage Bin:} Multiple toys are placed on the floor randomly. A big plastic storage bin is placed near the robot. The robot needs to clean up the floor by picking up the toys and throwing them into the bin. It can use one of the hind legs to drag the bin while reaching for the toy to improve efficiency (\textbf{D}).
% \item \textit{Chair:} The robot needs to move the chair through a narrow opening, where a small box is blocking this opening. The robot needs to clear up the box before dragging the chair through this opening. An efficient way is to hold the chair with the arm while using either the front or hind leg to move the box out of the way (\textbf{D}).
% \item \textit{Laundry Bag:} The robot needs to pick up a blanket from the ground and put it into the large laundry bag. The robot is requested to approach the blanket with one hind leg, dragging the laundry bag to improve efficiency (\textbf{D}).
% \end{itemize}
\section{Task Design}
\label{app:task}
This section details the real-world loco-manipulation tasks used to evaluate the \algabrvname framework, as introduced in \Cref{sec:experiments_highlevel}. As shown in \Cref{fig:execution}, we define a suite of twelve diverse and complex tasks, grouped into three categories based on the nature of interlimb coordination. Some tasks are instantiated in multiple variants tailored to different task interfaces, including \textbf{D} for direct target specification, \textbf{C} for contact-point-based control, and \textbf{L} for language instruction. For details about the task interfaces, please refer to \Cref{app:exp_task_interface}.
\subsection{Mobile Interlimb Coordination}
These tasks require the robot to manipulate large objects using its arm and one leg while navigating the space with the remaining three legs.
\textbf{Yoga Ball.} A 55\,cm yoga ball is placed on a platform in front of the robot. The robot is asked to either (1) pick up the ball and place it into a nearby laundry basket (\textbf{D}); or (2) pick up the ball and move backward with it while maintaining balance (\textbf{C}, \textbf{L}).
\textbf{Shipping Box.} A large cardboard box is placed randomly in the environment. The robot is required to pick up the box and transport it to a predefined location (\textbf{D}).
\subsection{Stationary Interlimb Coordination}
These tasks involve coordinated arm and leg motion for object manipulation while robustly balancing the torso with the remaining three legs.
\textbf{Tire Pump.} A small manual tire pump is connected to a flat bike tire. The robot stabilizes the pump using the arm and uses one leg to step on the pump repeatedly to inflate the tire (\textbf{D}).
\textbf{Trash Bin.} An eight-gallon step-on trash bin is placed in front of the robot. The robot is asked to either (1) pick up a Coke can, step on the pedal to open the lid, and throw the can into the bin (\textbf{D}); or (2) with the can already grasped, step on the pedal and dispose of the can (\textbf{C}).
\textbf{Deck Box.} A large deck box is placed in front of the robot, with a folded blanket on the ground. The robot grasps the blanket with its arm, lifts the lid of the deck box using a leg, and places the blanket inside while maintaining stability (\textbf{D}).
\textbf{Small Bin.} A one-gallon step-on bathroom bin is placed randomly. The robot picks up a paper ball from the floor, moves to the bin, opens the lid using one leg, and throws the ball inside using the arm (\textbf{D}).
\subsection{Foot-Assisted Manipulation}
These tasks are feasible with the arm alone but benefit from the use of a leg to enhance efficiency or robustness.
\textbf{Tool Chest.} A tool chest with one or more drawers is used in multiple variants: (1) pick up a blue tape from the floor, place it into an open drawer, and close the drawer using one leg (\textbf{D}); (2) place a pre-held tape into the drawer and close it with a leg (\textbf{C}); (3) simultaneously close two drawers using the arm and a leg (\textbf{L}).
\textbf{Storage Bin.} Toys are randomly scattered on the floor. The robot picks them up and throws them into a large plastic bin. To improve efficiency, it may drag the bin closer using a hind leg while reaching for toys (\textbf{D}).
\textbf{Chair.} The robot must move a chair through a narrow opening partially blocked by a box. It clears the box using a leg while simultaneously dragging the chair with its arm (\textbf{D}).
\textbf{Basket.} A shopping basket is placed on the floor. The robot must: (1) pick up a paper towel from a shelf, place it into the basket, and lift the basket using a leg (\textbf{D}); or (2) with a toy in hand, place it into the basket, then lift and carry the basket while moving backward (\textbf{C}).
\textbf{Drawer.} Multiple toys are scattered on the ground. The robot must clean up by picking up toys one by one and placing them on a small nightstand, opening and closing the drawer with a leg as needed (\textbf{D}).
\textbf{Laundry Bag.} The robot picks up a blanket from the ground and places it into a large laundry bag. To improve efficiency, it uses a hind leg to drag the laundry bag closer while grasping the blanket (\textbf{D}).
\section{Implementation Details of Reinforcement Learning}
\label{app:mdp}
This section outlines the implementation details of our reinforcement learning pipeline, especially regarding the Markov Decision Process (MDP) formulation used to train the reinforcement learning locomotion controller described in \Cref{subsec:rl}. The policy is trained in IsaacLab~\citep{mittal2023orbit} using Proximal Policy Optimization (PPO) as implemented in RSL\_RL~\citep{rudin2022learning}. All symbols follow the definitions provided in \Cref{tab:notations}.
\subsection{Observation Space}
\Cref{tab:obs} summarizes the observation terms used as policy input. The observation space consists of two main components: the robot state (proprioception) and the target commands.
The robot state includes base linear and angular velocities, gravity vector, joint positions and velocities, as well as the previous action. To improve generalization, we apply additive noise to these terms during training. The noise models are detailed in \Cref{tab:obs}. No scaling or clipping is applied to the individual observation terms.
The target commands specify desired robot behavior, including base linear and angular velocities, body attitude, and joint positions for the arm and legs. These commands are sampled from a pre-computed distribution designed to ensure physical feasibility and collision-free configurations. Linear and angular velocity commands are drawn from $\mathcal{U}(-1.0, 1.0)$ in $\mathrm{m/s}$ and $\mathrm{rad/s}$, respectively. Roll and pitch commands are sampled from $\mathcal{U}(-0.3, 0.3)$ radians, and base height from $\mathcal{U}(0.3, 0.7)$ meters.
To ensure feasibility and trackability, target commands are sampled and post-processed offline during training.
Joint targets are uniformly sampled within the configuration space and filtered using rejection sampling. Samples are discarded if they result in self-collision or ground collision. Additionally, we compute the support polygon and center of gravity (CoG) for each sampled configuration and retain only those in which the CoG lies within the support polygon. A total of $1.0 \times 10^7$ valid command samples are generated offline for training.
\begin{table}[!ht]
\centering
\caption{Observation terms and additive noise models used during training. No scaling or clipping is applied to individual terms.}
\label{tab:obs}
\begin{tabular}{lcc}
\toprule
\textbf{Observation Term} & \textbf{Symbol} & \textbf{Additive Noise} \\
\midrule
Base Linear Velocity & $\bm v_{B}$ & $\mathcal{U}(-0.1, 0.1)$ \\
\rowcolor[HTML]{EFEFEF} Base Angular Velocity & $\bm \omega_{B}$ & $\mathcal{U}(-0.2, 0.2)$ \\
Gravity Vector & $\bm g$ & $\mathcal{U}(-0.05, 0.05)$ \\
\rowcolor[HTML]{EFEFEF} Joint Positions & $\bm q$ & $\mathcal{U}(-0.05, 0.05)$ \\
Joint Velocities & $\dot{\bm q}$ & $\mathcal{U}(-0.5, 0.5)$ \\
\rowcolor[HTML]{EFEFEF} Previous Action & $\bm{a}$ & None \\
\midrule
Target Linear Velocity & $\bm v_B^*$ & None \\
\rowcolor[HTML]{EFEFEF} Target Angular Velocity & $\bm \omega_B^*$ & None \\
Target Joint Positions (Arm \& Leg) & $\bm q^*$ & None \\
\rowcolor[HTML]{EFEFEF} Target Base Orientation & $\bm x_{rot}^*$ & None \\
\bottomrule
\end{tabular}
\end{table}
\subsection{Reward Function}
\Cref{tab:rewards} summarizes the reward terms and their corresponding weights. To accelerate policy convergence, we adopt a reward scheduling strategy in which regularization terms (the lower half of \Cref{tab:rewards}) are initialized at zero and linearly increased over the first 10,000 training steps, after which they remain constant.
The gait reward enforces distinct locomotion patterns depending on the number of supporting legs. For four-leg locomotion, the policy is encouraged to produce a symmetric trotting gait. The reward is computed based on contact time $t_c^j$ and air time $t_a^j$ of each foot $j \in {\text{FL}, \text{FR}, \text{HL}, \text{HR}}$, following the formulation in~\citep{mittal2023orbit}. Specifically, we penalize mismatches in contact and air durations between diagonal foot pairs, and between each foot and the air duration of its non-diagonal counterpart:
\begin{equation}
\begin{aligned}
\label{eq:rew_gait_four}
r_{four-leg} = & || t_c^{FL} - t_c^{HR} || + || t_c^{FR} - t_c^{HL} || + || t_a^{FL} - t_a^{HR} || + || t_a^{FR} - t_a^{HL} || \\
+ & || t_c^{FL} - t_a^{FR} || + || t_c^{HL} - t_a^{HR} || + || t_a^{FL} - t_c^{FR} || + || t_a^{HL} - t_c^{HR} ||
\end{aligned}
\end{equation}
\begin{figure}
\centering
\includegraphics[width=0.6\linewidth]{figures/three_leg_phase.pdf}
\caption{
% Target contact time for each foot during three-leg locomotion.
\textbf{Gait regularization.} Desired contact timing among the remaining three supporting legs when one leg is lifted for manipulation. The phases are cyclically staggered to enable stable dynamic bouncing.
}
\label{fig:three_gait_phase}
\end{figure}
For three-leg locomotion, we enforce a cyclic bouncing gait among the remaining supporting feet. The target contact phases are visualized in \Cref{fig:three_gait_phase}. When one leg (\eg, FL) is lifted for manipulation, the remaining feet (FR, HR, and HL) are assigned to \textit{foot 1}, \textit{foot 2}, and \textit{foot 3} respectively in clockwise order. A fixed gait cycle duration of $T_{\text{gait}} = 0.4$ seconds is used to define desired contact timing. The corresponding reward penalizes deviation from the target phase alignment:
\begin{equation}
\begin{aligned}
\label{eq:rew_gait_three}
r_{three-leg} = & \mathbbm{1}_{t_a^1 > 0} \cdot (|| t_c^{2} - t_a^{1} - \frac{T_{gait}}{3} || + || t_c^{3} - t_a^{1} ||)
\end{aligned}
\end{equation}
% The corresponding weights are in \Cref{tab:rewards}. In practice, we applied a reward schedule to speed up the policy convergence. The regularization terms, the second half in \Cref{tab:rewards}, are initially set to 0 and increase linearly as the training proceeds. They reaches the value in the table at 10,000 steps and remain static afterwards.
% For the gait reward, we enforce a trotting gait during four-leg locomotion and a three-phase bouncing gait during three-leg locomotion. The gait reward is implemented based on contact time $t_{c}^j$ and air time $t_{a}^j$ of each foot $j\in\{
% FL, FR, HL, HR \}$, as defined in~\citep{mittal2023orbit}. During four-leg trotting, we minimize the error between contact and air time of the diagonal feet. Moreover, we enforce the contact time of each foot to be the same as its non-diagonal foot's air time:
% \begin{equation}
% \begin{aligned}
% \label{eq:rew_gait_four}
% r_{four-leg} = & || t_c^{FL} - t_c^{HR} || + || t_c^{FR} - t_c^{HL} || + || t_a^{FL} - t_a^{HR} || + || t_a^{FR} - t_a^{HL} || \\
% + & || t_c^{FL} - t_a^{FR} || + || t_c^{HL} - t_a^{HR} || + || t_a^{FL} - t_c^{FR} || + || t_a^{HL} - t_c^{HR} ||
% \end{aligned}
% \end{equation}
% \xz{add the figure for three-leg gait, specify foot 1, 2, 3 in the plot instead of the specific foot name}
% During three-leg bouncing, the desired contact phase for each foot is illustrated in \Cref{fig:three_gait_phase}. When one foot is lifted for manipulation, the remaining three feet are assigned to different phases in the clockwise order. For example, when the FL leg is lifted, the FR foot is assigned to \textit{foot 1}, the HR is assigned to \textit{foot 2}, and HL is assigned to \textit{foot 3}. We also defined a cycle time $T_{gait}=0.4s$ to specify the time of the contact cycle. The gait reward during three-leg locomotion is:
% \begin{equation}
% \begin{aligned}
% \label{eq:rew_gait_three}
% r_{three-leg} = & \mathbbm{1}_{t_a^1 > 0} \cdot (|| t_c^{2} - t_a^{1} - \frac{T_{gait}}{3} || + || t_c^{3} - t_a^{1} ||)
% \end{aligned}
% \end{equation}
\begin{table}[t]
\centering
\caption{Reward Terms Summary. The environment scales the reward weights with the time-step $dt$~\cite{rudin2022learning}. For brevity, we drop the time-step $t$ from individual quantities.}
\label{tab:rewards}
\begin{tabular}{lcc}
\toprule
\textbf{Term Name} & \textbf{Definition} & \textbf{Weight} \\
\midrule
Robot Base Linear Velocity Tracking & $\exp{-\frac{|| \bm{v}_{B} - \bm{v}^*_{B} ||}{0.25}}$ & $7.0$ \\
\rowcolor[HTML]{EFEFEF} Robot Base Angular Velocity Tracking & $\exp{-\frac{|| \bm \omega_{B} - \bm \omega^*_{B} ||}{0.25}}$ & $3.5$ \\
Robot Base Attitude Tracking & $||\bm{x}_{rot} - \bm{x}_{rot}^*||^2$ & $-120$ \\
\midrule
\rowcolor[HTML]{EFEFEF} Gait & $ \mathbbm{1}_{walking}\cdot (r_{four-leg} + r_{three-leg})$ & $-5$ \\
Robot Joint Velocity & $||\dot{\bm q}||^2$ & $-1.0 \times 10^{-5}$ \\
\rowcolor[HTML]{EFEFEF} Robot Joint Acceleration & $||\ddot{\bm q}||^2$ & $-1.0 \times 10^{-6}$ \\
Robot Applied Joint Torque & $||{\bm \tau}||^2$ & $-2.0 \times 10^{-4}$ \\
\rowcolor[HTML]{EFEFEF} Action Rate & $||\dot{\bm a}||^2$ & $-0.1$ \\
Flight Penalty & $\mathbbm{1}_{flight}$ & $-10.0$ \\
\bottomrule
\end{tabular}
\end{table}
\subsection{Termination and Reset}
During training, an episode terminates either when the agent reaches a time limit of 20 seconds or when the robot's body makes contact with the ground. Upon termination, the environment resets the robot to a randomly sampled initial state.
%
The base height is sampled from $\mathcal{U}(0.2, 1.3)$ meters, while roll and pitch are sampled from $\mathcal{U}(-0.5, 0.5)$ radians. The $xy$ position and yaw are uniformly sampled across the workspace. Initial base linear and angular velocities are drawn from $\mathcal{U}(-0.5, 0.5)$ in both $\mathrm{m/s}$ and $\mathrm{rad/s}$, respectively. Joint positions for both the arm and legs are uniformly sampled within their respective configuration spaces.
% During the training, the episode is terminated if the agent reaches a timeout of $20s$ or the body contact with the ground. Once terminated, the robot is randomly reset to its initial state distribution. The base is sampled with height $\mathcal{U}(0.2, 1.3)$ in meters, the roll and pitch are sampled with $\mathcal{U}(-0.5, 0.5)$ in radius. The $xy$ position and yaw are sampled uniformly in the whole world. Moreover, we sample initial base linear and angular velocities within the range of $\mathcal{U}(-0.5, 0.5)$ in $m/s$ and $rad/s$. The joint positions for the arm and legs are sampled uniformly within the configuration space.
% \begin{table}[ht]
% \centering
% \caption{MDP Hyperparameters.}
% \label{tab:adp}
% \begin{tabular}{l|c}
% \toprule
% \textbf{Hyperparameter} & \textbf{Value} \\
% \midrule
% Episode length & $\SI{15}{s}$ \\
% \rowcolor[HTML]{EFEFEF} Simulation time-step & $\SI{0.005}{s}$ \\
% Control decimation & 4 \\
% \midrule
% \rowcolor[HTML]{EFEFEF} Residual phase action scale: $\sigma_1$ & 0.01 \\
% Robot action scale: $\sigma_2$ & 0.5 \\
% \midrule
% \rowcolor[HTML]{EFEFEF} Adaptive phase~\eqref{eq:phase_rate_reference}: $w_1$ & -0.2 \\
% Adaptive phase~\eqref{eq:phase_rate_reference}: $w_2$ & -0.2 \\
% \rowcolor[HTML]{EFEFEF} Adaptive phase~\eqref{eq:phase_rate_reference}: $w_4$ & -0.2 \\
% Adaptive phase~\eqref{eq:phase_rate_reference}: $\lambda$ & -50.0 \\
% \rowcolor[HTML]{EFEFEF} Adaptive phase~\eqref{eq:phase_rate_reference}: $k$ & 10.0 \\
% \bottomrule
% \end{tabular}
% \end{table}
% \subsection{Learning Algorithm}
% We train the policy use the on-policy RL algorithm, Proximal Policy Optimization (PPO). The .....
% %
% A complete list of hyperparameters and their values is specified in \Cref{tab:ppo-hp}.
% \subsection{Training procedures}
% \begin{table}[ht]
% \centering
% \vspace{-13pt}
% \caption{PPO Hyperparameters}
% \label{tab:ppo-hp}
% \begin{tabular}{l|c}
% \toprule
% \textbf{Hyperparameter} & \textbf{Value} \\
% \midrule
% Empirical Normalization & True \\
% \rowcolor[HTML]{EFEFEF} Learning Rate (start of training) & 1e-3 \\
% Learning Rate Schedule & ``adaptive" (based on KL-divergence~\cite{rudin2022learning})\\
% \rowcolor[HTML]{EFEFEF} Discount Factor & 0.99 \\
% GAE Discount Factor & 0.95 \\
% \rowcolor[HTML]{EFEFEF} Desired KL-divergence & 0.01 \\
% Clip Range & 0.2 \\
% \rowcolor[HTML]{EFEFEF} Entropy Coefficient & 0.0 \\
% Value Function Loss Coefficient & 1.0 \\
% \rowcolor[HTML]{EFEFEF} Batch Size & 245,760 $(4096 \times 60)$ \\
% Mini-Batch Size & 61,440 $(4096 \times 15)$ \\
% \rowcolor[HTML]{EFEFEF} Number of Epochs & 5 \\
% Number of iterations & 10,000 \\
% \bottomrule
% \end{tabular}
% \end{table}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{Implementation Details of Sim-to-Real Transfer}
\label{app:sim_to_real}
This section details the techniques used to enable successful transfer of the learned locomotion policy from simulation to the real robot. We describe the domain randomization strategies employed during training, followed by the motor calibration procedure that accounts for discrepancies in actuator dynamics between simulation and hardware.
\subsection{Domain Randomization}
To enhance the robustness of the locomotion policy under real-world variability, we apply extensive domain randomization during training. Specifically, we randomize physical properties such as link and actuator friction coefficients, actuator stiffness and damping parameters, and terrain mesh geometry. In addition, we apply periodic random external pushes to the base to simulate unexpected disturbances.
\subsection{Motor Calibration}
\begin{figure}
\centering
\includegraphics[width=0.5\linewidth]{figures/sim2real.pdf}
\caption{
% Measured torque versus joint velocity for knee joints in the real world. Each blue point represents a sensor-recorded timestep, while the red lines indicate the initial torque limits.
\textbf{Torque-Velocity Calibration.} Empirical torque versus joint velocity measurements for the knee joints, collected on the real robot. Each blue point corresponds to a sensor-recorded sample. The red lines denote the initial uncalibrated torque limits used in simulation.
}
\label{fig:sim2real}
\end{figure}
As discussed in \Cref{subsec:sim2real}, accurate actuator torque limits are critical for successful zero-shot deployment on hardware. In practice, we found that calibrating only the knee joints using torque-velocity constraints is sufficient. \Cref{fig:sim2real} presents the torque-velocity profiles collected from the initial policy deployed on the real robot.
The calibration process begins with manual estimation of joint limits, followed by automatic refinement using CMA-ES~\citep{nomura2024cmaes}. To optimize the limits, we replay the same policy command sequence in simulation and minimize the Wasserstein distance between the rollout trajectories from the real world and simulation. In our experiments, a single round of this procedure yielded satisfactory results.
% As illustrated in \Cref{subsec:sim2real}, torque limits for actuators are essential for zero-shot real world deployment. In practice, we found it's sufficient to calibrate the knee joints with torque-velocity limits. \Cref{fig:sim2real} shows the collected torque-velocity data with our initial policy. The limits are first measured manually and further optimized with CMA-ES. The command sequence is replayed in the simulation with the same policy, and the limits are optimized to minimize the Wasserstein distance between the real-world rollout trajectory and simulation rollouts. We found one round of the procedure sufficient.
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{Implementation Details of Task Interface}
\label{app:exp_task_interface}
This section provides implementation details for the three task interface modalities used in our experiments, including direct target specification, contact-point-based decomposition, and language-driven task grounding via vision-language models (VLMs). These interfaces correspond to the modalities described in \Cref{subsec:task_interface} and are used to generate the target-driven task representation for the \algabrvname controller.
\subsection{Direct Targets}
\label{app:direct_targets}
% The targets required for command-level control in \Cref{sec:experiments_lowlevel} include the torso and end effector targets. We use a Sony PS5 joystick to set torso velocity targets and 3D spacemouse devices to set separate target poses for each of the selected end-effectors.
The \textit{Direct Target} interface supports teleoperation by specifying targets for selected limbs and the torso in real time. This is achieved through a compact teleoperation setup that combines off-the-shelf input devices for intuitive control. A Sony PS5 joystick is used to control the robot's torso velocity, allowing for responsive and agile base movement. For manipulation, we use a 3Dconnexion SpaceMouse to set precise 6-DoF pose targets for the arm or any of the legs, enabling full spatial control over end-effector positioning and orientation. These inputs are sampled at a fixed rate and interpolated into smooth Cartesian-space trajectories using linear and spherical interpolation.
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\subsection{Contact Points}
\label{app:contact_points}
In the contact point based interface, we use a contact-based task decomposition to specify the contact points for different end-effectors to solve the tasks. We specify the target contact point on the object, the pre- and post-contact directions, and the selected limb (arm or one of the legs).
\textbf{Contact Point Selection.}
At the start of stage $i$, the system fuses RGB-D images from the camera sensors with the robot's proprioceptive data to produce a 3D point cloud observation $\prescript{w}{}{P}^{(i)}\in\mathbb{R}^{n\times 3}$, where $n$ is the total number of points. From this point cloud, we derive the corresponding targets $\mathcal{T}^{(i)}$ for the current stage by selecting two key points(when we use two limbs for manipulation): one for the desired 3D location of the arm end-effector $\prescript{w}{}{x}^{(i)}_\text{arm}\in\prescript{w}{}{P}^{(i)}$, and another for the desired 3D location of the foot $\prescript{w}{}{x}^{(i)}_\text{leg}\in\prescript{w}{}{P}^{(i)}$. The desired $SE(3)$ pose of the end-effector, $\prescript{w}{}{p}^{(i)}_\text{arm}=\begin{bmatrix}
\prescript{w}{}{x}^{(i)}_\text{arm}&\prescript{w}{}{R}^{(i)}_\text{arm}
\end{bmatrix}^\top$, combines the chosen position $\prescript{w}{}{x}^{(i)}_\text{arm}$ with a specified rotation $\prescript{w}{}{R}^{(i)}_\text{arm}\in\mathbb{R}^3$. Meanwhile, the desired foot location is simply $\prescript{w}{}{p}^{(i)}_\text{leg}=\prescript{w}{}{x}^{(i)}_\text{leg}$. The inverse kinematics (IK) module then calculates the desired body pose $\prescript{w}{}{p}^{(i)}_\text{body}=\rm{IK}(\prescript{w}{}{p}^{(i)}_\text{arm},\prescript{w}{}{p}^{(i)}_\text{leg})$. Additionally, we specify a gripper state $g^{(i)}$ and a desired leg mask $m^{(i)}$, resulting in the complete set of stage targets $\mathcal{T}^{(i)}$.
% In practice, these targets can be determined either by user annotation or automatically via vision-language models (VLMs).
As shown in \Cref{fig:app:3dpointcloud}, the interactive point-cloud selection interface will appear every time the robot has reached the previous targets, allowing the user to click on point cloud $\prescript{w}{}{P}^{(i)}$ and specify $\prescript{w}{}{x}^{(i)}_\text{arm, leg}$ and $\prescript{w}{}{R}^{(i)}_\text{arm}$, followed by manual input of $g^{(i)}$ and $m^{(i)}$. An autonomous VLM agent can also replace the selection from human annotation, as introduced in \Cref{app:language_intstructions}.
\begin{figure}
\centering
\includegraphics[width=1.0\linewidth]{figures/3dpointcloud.pdf}
\caption{
% Constructed 3d point cloud using Spot onboard stero cameras in contact point interface. The user is asked to select contact points, pre- and post-contact directions from the point cloud.
\textbf{Point Cloud for Contact Point Interface.} Point cloud constructed using Spot’s onboard stereo cameras in the contact point interface. Users annotate the scene by selecting arm and leg contact points, along with pre- and post-contact directions.
}
\label{fig:app:3dpointcloud}
\end{figure}
\textbf{Waypoints Generation.}
Once the targets $\mathcal{T}^{(i)}$ are specified, the next step is to generate dense waypoints guiding the robot from its current pose to the desired configuration. While any off-the-shelf trajectory optimization (TO) algorithm could be used, here we employ a straightforward approach: linear interpolation for 3D translation and spherical linear interpolation (Slerp) for 3D rotation. Specifically, we produce three separate trajectories for each target set $\mathcal{T}^{(i)}$: the arm end-effector trajectory $\prescript{w}{}{\xi}^{(i)}_\text{arm}\in\mathbb{R}^{H\times6}$, the leg end-effector trajectory $\prescript{w}{}{\xi}^{(i)}_\text{leg}\in\mathbb{R}^{H\times3}$, and the torso trajectory $\prescript{w}{}{\xi}^{(i)}_\text{torso}\in\mathbb{R}^{H\times6}$.
Based on the current pose, the target pose, and the maximum linear and angular velocities, we determine the required number of waypoints for each trajectory (\textit{i.e.}, $H_\text{arm}$, $H_\text{leg}$, and $H_\text{torso}$). The overall trajectory length $H$ is then defined as $H=\max\left(H_\text{arm}, H_\text{leg}, H_\text{torso}\right)$.
Following these trajectories, the robot tracks each successive waypoint at every timestep using the low-level RL controller described in the next section. When all trajectories have been executed, we check whether the difference between the robot’s final pose and the target configuration is below predefined thresholds. If it is, the targets $\mathcal{T}^{i}$ are considered “reached”; otherwise, a new set of trajectories is generated from the current pose.
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\begin{figure}
\centering
\includegraphics[width=\linewidth]{figures/vlm_pipeline.pdf}
\caption{
% \textbf{VLM-based contact selection.} Arm and Leg contact points are automatically selected using VLM queries.
% \textbf{Language Instruction Task Interface.} Given the free-form language instruction specified by the user, contact points for the arm and leg are automatically selected using GPT-4o queries grounded in object masks produced by SAM2.
\textbf{Language Instruction Task Interface.} Overview of the vision-language pipeline used to automatically infer end-effector contact points from free-form task descriptions. Given an RGB observation, SAM2 segments the scene into object masks, which are then passed to GPT-4o for semantic reasoning. The model selects relevant masks based on task semantics, identifies keypoint candidates, and outputs final contact points for the arm and leg. Examples are shown for the two tasks, \textit{Yoga Ball} (top row) and \textit{Tool Chest} (bottom row).
}
\label{fig:app:vlm}
\end{figure}
\clearpage
\subsection{Language Instructions}
\label{app:language_intstructions}
The \textit{Language Instruction} interface enables natural, flexible task specification by grounding free-form language to contact points via vision-language reasoning.
%
We use GPT-4o~\citep{openai2024gpt4ocard} to process RGB images captured from the robot's front camera. The scene is first segmented into object masks using SAM2~\citep{ravi2024sam2}. In the first stage, GPT-4o selects the task-relevant masks from among these segments, identifying objects that are directly involved in the task. In the second stage, keypoints are sampled within the selected regions, and GPT-4o selects appropriate contact points for the arm and leg end-effectors based on the task description and robot capabilities.
The complete vision-language grounding pipeline is illustrated in \Cref{fig:app:vlm}. Predefined contact directions are assigned according to task semantics. For instance, in the \textit{Yoga Ball} task, the arm and leg move toward each other. In the \textit{Tool Chest} task, both end-effectors move outward from the robot body toward the object.
% The main difference between language instruction and contact point interface is the way we specify the task-specific contact points and directions. We use GPT-4~\citep{openai2024gpt4ocard} to \emph{1)} select task-related masks from SAM2~\citep{ravi2024sam2} and \emph{2)} select contact points for limbs (arm and leg based on sampled key points in the selected mask.
% The full pipeline of contact point selection using VLM is shown in \Cref{fig:app:vlm}. We first get the raw RGB image from the Spot front camera and use SAM2 to segment it into uncategorized masks. The numbered masks are sent to GPT4o to select the task-related masks. Key points are then sampled inside the selected masks and numbered. The labeled image is then sent to GPT4o for contact point selection for all end-effectors. For the two tasks we show with the language instruction interface, we set the pre- and post-contact directions using pre-defined relative directions: for \textit{yoga-ball}, both end-effectors move in the direction approaching the other end-effector; for \textit{tool chest}, the end-effectors move in the direction from the body to the target points.
\textbf{Example task-specific prompts.} Below are the prompt template for language instruction interfaces.
\begin{tcolorbox}[
rounded corners,
title=Prompt for task-specific mask selection
]
I'm showing you an image where different objects have been segmented and numbered.\\
Based on this task description:\\
Task details:\\
Using the arm and leg to close the opening tool chest drawers.\\
Objects of interest: tool chest drawers.\\
Robot capabilities:\\
- Leg: end effector provide contact with objects\\
- Arm: end effector provide contact with objects\\
Constraints:\\
- Use the arm to close the upper opening drawer of the tool chest\\
- Use the leg to close the lower opening drawer of the tool chest\\
- ground is not related to the task, so you should ignore it when choosing the contact points\\
\\
Please identify which numbered regions/objects are most relevant for this task. Choose the most related masks.\\
Respond with ONLY the number of the relevant mask in this format:\\
MASK: \textit{mask number}\\
DO NOT include any explanation or reasoning.
\end{tcolorbox}
\begin{tcolorbox}[
rounded corners,
title=Prompt for contact points selection based on provided key points,
breakable
]
Based on this task: ``Tool Chest Closing Task'', select TWO points for the robot:
1. One point where the ARM/GRIPPER should contact the object\\
2. One point where the LEG should contact or interact with the object\\
\\
Task details:\\
Using the arm and leg to close the opening tool chest drawers.\\
Objects of interest: tool chest drawers.\\
Robot capabilities:\\
- Leg: end effector provide contact with objects\\
- Arm: end effector provide contact with objects\\
Constraints:\\
- Use the arm to close the upper opening drawer of the tool chest\\
- Use the leg to close the lower opening drawer of the tool chest\\
- ground is not related to the task, so you should ignore it when choosing the contact points\\
\\
Choose points that will allow for stable and effective manipulation.\\
\end{tcolorbox}
After the contact points are selected, the target and waypoints generation procedure is the same as the implementation in the contact point interface.
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% \subsection{Details on Baseline Methods}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% \subsection{Detailed Failure Analysis}
% \begin{itemize}
% \item Yoga Ball
% %1 Success
% %2 Success
% %3 Failure - Kick Ball
% %4 Success
% %5 Failure - Unbalance
% %6 Success
% %7 Success
% %8 Failure - Ball Slip
% %9 Success
% %10 Success
% \item Tool Chest
% %1 Success
% %2 Success
% %3 Success
% %4 Success
% %5 Success
% %6 Success
% %7 Failure - Controller Off track
% %8 Success
% %9 Success
% %10 Success
% \item Trash Bin
% %1 Success
% %2 Success
% %3 Success
% %4 Success
% %5 Success
% %6 Failure - Perception Error
% %7 Success
% %8 Success
% %9 Success
% %10 Success
% \item Basket
% %1 Success (bag flip)
% %2 Failure - Leg pose error
% %3 Failure - Perception error
% %4 Success
% %5 Success (bag flip)
% %6 Success
% %7 Success (timestamp too far in the past in the end)
% %8 Success (timestamp too far in the past in the end)
% %9 Success (timestamp too far in the past in the end)
% %10 Success
% %11 Success
% \end{itemize}
\begin{figure}
\centering
\includegraphics[width=\linewidth]{figures/tracking_error_ablation.pdf}
\caption{\textbf{Ablation Study on Motor Calibration.} Comparison of leg and arm end-effector co-tracking performance between the full \textit{ReLIC} policy and ablated variants. Removing motor calibration (\textit{ReLIC w/o Motor Calib.}) leads to reduced tracking accuracy and stability.
% Omitting gait regularization (\textbf{ReLIC w/o Gait Regular.}) results in complete control failures during the evaluation tasks.
}\label{fig:app:no_calibration_ablation}
\end{figure}
\begin{figure}
\centering
\includegraphics[width=\linewidth]{figures/gait_ablation.pdf}
\caption{\textbf{Ablation Study on Gait transition.} Gait transition for \textit{ReLIC w/o Gait Regular.}: compared to \textit{ReLIC}, the robot has a bouncing gait for both four-legged and three-legged motion, which provides less stability in both tracking and gait switching. Videos can be found on the website.}
\label{fig:app:no_gait_ablation}
\vspace{-0.1cm}
\end{figure}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{Ablation Study}
\label{app:experimexp_quantents}
Below we discuss the evaluation of the \algabrvname controller reported in \Cref{sec:experiments_lowlevel}.
%
We present two ablation experiments on the locomotion reinformcement learning controller trained in the simulator. Both experiments are performed using the same reward structure and training setup in simulation.
Specifically, we evaluate two ablated variants: \textbf{ReLIC w/o Motor Calib.}, trained without motor calibration, and \textbf{ReLIC w/o Gait Regular.}, trained without the gait regularization reward.
% \subsection{Motor Calibration}
% \label{app:exp_eef_motion}
We assess the importance of motor calibration by measuring the target tracking accuracy of end-effector trajectories for both the arm and legs. As shown in \Cref{fig:app:no_calibration_ablation}, motor calibration substantially improves the stability and precision. The calibrated policy maintains smoother and more synchronized movements across limbs.
%
Furthermore, the calibrated controller demonstrates significantly improved robustness. The robot can remain statically balanced on three legs for over 10 minutes without requiring active adjustment. In contrast, the uncalibrated policy struggles to maintain balance, frequently initiating compensatory motions. A video of the 10-minute static standing demonstration is available on the project website.
% \subsection{Gait Regularization}
% \label{app:exp_gait_tran}
We examine the effect of removing the gait regularization term during training by applying the same gait transition protocol used in \Cref{fig:gait_switch} to the \textbf{ReLIC} policy.
The resulting gait behaviors are visualized in \Cref{fig:app:no_gait_ablation}.
%
Without the regularization, the learned gait degenerates into a repetitive bouncing motion, both in quadrupedal and tripedal modes. Despite using motor calibration, this policy exhibits poor stability and consistently fails during the rhombus-shaped trajectory tracking task due to falls. These results highlight that motor calibration alone is insufficient; structured gait priors are critical for achieving stable, flexible locomotion across contact configurations.
% \subsection{Motor Calibration Ablation}
% \label{app:exp_eef_motion}
% We show the effectiveness of motor calibration for policy robustness by comparing the end-effectors motion co-tracking performance with and without the motor calibration. The results are shown in \Cref{fig:app:no_calibration_ablation}.
% The tracking is much more stable for both end-effectors after the motor calibration.
% In addition, the calibrated policy is more robust during deployment. The robot is able to stand still with three legs for longer than 10 minutes after calibration, while the uncalibrated one need to move to balance itself occasionally. See the recorded 10 min standing video on the website.
% \subsection{Gait Regularization Ablation}
% \label{app:exp_gait_tran}
% We show the effectiveness of adding gait regularization during training by doing same gait transition experiment like \Cref{fig:gait_switch} on \textit{ReLIC-nogait} policy. The gaits are shown in \Cref{fig:app:no_gait_ablation}. The gait emerged from training without explicit gait regularization is \textit{bouncing} for both four-leg and three-leg movements. Although motor calibration is used for this policy, it's less stable and cannot finish the rhombus-shaped tracking tasks without falling down. For a stable locomotion policy with flexible gaits, both motor calibration and gait regularization are essential.
\begin{thebibliography}{54}
\providecommand{\natexlab}[1]{#1}
\providecommand{\url}[1]{\texttt{#1}}
\expandafter\ifx\csname urlstyle\endcsname\relax
\providecommand{\doi}[1]{doi: #1}\else
\providecommand{\doi}{doi: \begingroup \urlstyle{rm}\Url}\fi
\bibitem[Rehman et~al.(2016)Rehman, Focchi, Lee, Dallali, Caldwell, and Semini]{rehman2016mobilemanipulator}
B.~U. Rehman, M.~Focchi, J.~Lee, H.~Dallali, D.~G. Caldwell, and C.~Semini.
\newblock Towards a multi-legged mobile manipulator.
\newblock In \emph{2016 IEEE International Conference on Robotics and Automation (ICRA)}, pages 3618--3624, 2016.
\newblock \doi{10.1109/ICRA.2016.7487545}.
\bibitem[Sleiman et~al.(2023)Sleiman, Farshidian, and Hutter]{sleiman2023versatile}
J.-P. Sleiman, F.~Farshidian, and M.~Hutter.
\newblock Versatile multicontact planning and control for legged loco-manipulation.
\newblock \emph{Science Robotics}, 8\penalty0 (81):\penalty0 eadg5014, 2023.
\bibitem[Gu et~al.(2025)Gu, Li, Shen, Yu, Xie, McCrory, Cheng, Shamsah, Griffin, Liu, Kheddar, Peng, Zhu, Shi, Nguyen, Cheng, Gao, and Zhao]{gu2025humanoidlocomotionmanipulationcurrent}
Z.~Gu, J.~Li, W.~Shen, W.~Yu, Z.~Xie, S.~McCrory, X.~Cheng, A.~Shamsah, R.~Griffin, C.~K. Liu, A.~Kheddar, X.~B. Peng, Y.~Zhu, G.~Shi, Q.~Nguyen, G.~Cheng, H.~Gao, and Y.~Zhao.
\newblock Humanoid locomotion and manipulation: Current progress and challenges in control, planning, and learning, 2025.
\newblock URL \url{https://arxiv.org/abs/2501.02116}.
\bibitem[Zimmermann et~al.(2021)Zimmermann, Poranne, and Coros]{zimmermann2021go}
S.~Zimmermann, R.~Poranne, and S.~Coros.
\newblock Go fetch!-dynamic grasps using boston dynamics spot with external robotic arm.
\newblock In \emph{2021 IEEE International Conference on Robotics and Automation (ICRA)}, pages 4488--4494. IEEE, 2021.
\bibitem[Hooks et~al.(2020)Hooks, Ahn, Yu, Zhang, Zhu, Chae, and Hong]{hooks2020alphred}
J.~Hooks, M.~S. Ahn, J.~Yu, X.~Zhang, T.~Zhu, H.~Chae, and D.~Hong.
\newblock Alphred: A multi-modal operations quadruped robot for package delivery applications.
\newblock \emph{IEEE Robotics and Automation Letters}, 5\penalty0 (4):\penalty0 5409--5416, 2020.
\newblock \doi{10.1109/LRA.2020.3007482}.
\bibitem[Bellicoso et~al.(2019)Bellicoso, Kr{\"a}mer, St{\"a}uble, Sako, Jenelten, Bjelonic, and Hutter]{bellicoso2019alma}
C.~D. Bellicoso, K.~Kr{\"a}mer, M.~St{\"a}uble, D.~Sako, F.~Jenelten, M.~Bjelonic, and M.~Hutter.
\newblock Alma-articulated locomotion and manipulation for a torque-controllable robot.
\newblock In \emph{2019 International conference on robotics and automation (ICRA)}, pages 8477--8483. IEEE, 2019.
\bibitem[Fu et~al.(2023)Fu, Cheng, and Pathak]{fu2023deep}
Z.~Fu, X.~Cheng, and D.~Pathak.
\newblock Deep whole-body control: learning a unified policy for manipulation and locomotion.
\newblock In \emph{Conference on Robot Learning}, pages 138--149. PMLR, 2023.
\bibitem[Zhang et~al.(2024)Zhang, Liang, Chen, Ze, and Xu]{zhang2024catch}
Y.~Zhang, T.~Liang, Z.~Chen, Y.~Ze, and H.~Xu.
\newblock Catch it! learning to catch in flight with mobile dexterous hands.
\newblock \emph{arXiv preprint arXiv:2409.10319}, 2024.
\bibitem[Bajracharya et~al.(2024)Bajracharya, Borders, Cheng, Helmick, Kaul, Kruse, Leichty, Ma, Matl, Michel, et~al.]{bajracharya2024demonstrating}
M.~Bajracharya, J.~Borders, R.~Cheng, D.~Helmick, L.~Kaul, D.~Kruse, J.~Leichty, J.~Ma, C.~Matl, F.~Michel, et~al.
\newblock Demonstrating mobile manipulation in the wild: A metrics-driven approach.
\newblock \emph{arXiv preprint arXiv:2401.01474}, 2024.
\bibitem[Fu et~al.(2025)Fu, Zhao, and Finn]{fu2024mobile}
Z.~Fu, T.~Z. Zhao, and C.~Finn.
\newblock Mobile aloha: Learning bimanual mobile manipulation using low-cost whole-body teleoperation.
\newblock In \emph{Proceedings of The 8th Conference on Robot Learning}, volume 270 of \emph{Proceedings of Machine Learning Research}, pages 4066--4083. PMLR, 06--09 Nov 2025.
\bibitem[Hu et~al.(2022)Hu, Stone, and Mart{\'\i}n-Mart{\'\i}n]{hu2023causal}
J.~Hu, P.~Stone, and R.~Mart{\'\i}n-Mart{\'\i}n.
\newblock Causal policy gradient for whole-body mobile manipulation.
\newblock In \emph{Proceedings of Robotics: Science and Systems}, 2022.
\bibitem[Kindle et~al.(2020)Kindle, Furrer, Novkovic, Chung, Siegwart, and Nieto]{kindle2020whole}
J.~Kindle, F.~Furrer, T.~Novkovic, J.~J. Chung, R.~Siegwart, and J.~Nieto.
\newblock Whole-body control of a mobile manipulator using end-to-end reinforcement learning.
\newblock \emph{arXiv preprint arXiv:2003.02637}, 2020.
\bibitem[Lew et~al.(2023)Lew, Singh, Prats, Bingham, Weisz, Holson, Zhang, Sindhwani, Lu, Xia, et~al.]{lew2023robotic}
T.~Lew, S.~Singh, M.~Prats, J.~Bingham, J.~Weisz, B.~Holson, X.~Zhang, V.~Sindhwani, Y.~Lu, F.~Xia, et~al.
\newblock Robotic table wiping via reinforcement learning and whole-body trajectory optimization.
\newblock In \emph{2023 IEEE International Conference on Robotics and Automation (ICRA)}, pages 7184--7190. IEEE, 2023.
\bibitem[Sun et~al.(2022)Sun, Orbik, Devin, Yang, Gupta, Berseth, and Levine]{sun2022fully}
C.~Sun, J.~Orbik, C.~M. Devin, B.~H. Yang, A.~Gupta, G.~Berseth, and S.~Levine.
\newblock Fully autonomous real-world reinforcement learning with applications to mobile manipulation.
\newblock In \emph{Conference on Robot Learning}, pages 308--319. PMLR, 2022.
\bibitem[Yang et~al.(2024)Yang, Kim, Hendrix, Kembhavi, Wang, and Ehsani]{yang2024harmonic}
R.~Yang, Y.~Kim, R.~Hendrix, A.~Kembhavi, X.~Wang, and K.~Ehsani.
\newblock Harmonic mobile manipulation.
\newblock In \emph{2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)}, pages 3658--3665. IEEE, 2024.
\bibitem[Wu et~al.(2023)Wu, Antonova, Kan, Lepert, Zeng, Song, Bohg, Rusinkiewicz, and Funkhouser]{wu2023tidybot}
J.~Wu, R.~Antonova, A.~Kan, M.~Lepert, A.~Zeng, S.~Song, J.~Bohg, S.~Rusinkiewicz, and T.~Funkhouser.
\newblock Tidybot: Personalized robot assistance with large language models.
\newblock \emph{Autonomous Robots}, 47\penalty0 (8):\penalty0 1087--1102, 2023.
\bibitem[He et~al.(2025)He, Luo, He, Xiao, Zhang, Zhang, Kitani, Liu, and Shi]{he2024omnih2o}
T.~He, Z.~Luo, X.~He, W.~Xiao, C.~Zhang, W.~Zhang, K.~M. Kitani, C.~Liu, and G.~Shi.
\newblock Omnih2o: Universal and dexterous human-to-humanoid whole-body teleoperation and learning.
\newblock In \emph{Proceedings of The 8th Conference on Robot Learning}, volume 270 of \emph{Proceedings of Machine Learning Research}, pages 1516--1540. PMLR, 06--09 Nov 2025.
\bibitem[Fu et~al.(2025)Fu, Zhao, Wu, Wetzstein, and Finn]{fu2024humanplus}
Z.~Fu, Q.~Zhao, Q.~Wu, G.~Wetzstein, and C.~Finn.
\newblock Humanplus: Humanoid shadowing and imitation from humans.
\newblock In \emph{Proceedings of The 8th Conference on Robot Learning}, volume 270 of \emph{Proceedings of Machine Learning Research}, pages 2828--2844. PMLR, 06--09 Nov 2025.
\bibitem[Ze et~al.(2024)Ze, Chen, Wang, Chen, He, Yuan, Peng, and Wu]{ze2024generalizable}
Y.~Ze, Z.~Chen, W.~Wang, T.~Chen, X.~He, Y.~Yuan, X.~B. Peng, and J.~Wu.
\newblock Generalizable humanoid manipulation with improved 3d diffusion policies.
\newblock \emph{arXiv preprint arXiv:2410.10803}, 2024.
\bibitem[Lu et~al.(2024)Lu, Cheng, Li, Yang, Ji, Yuan, Yang, Yi, and Wang]{lu2024mobile}
C.~Lu, X.~Cheng, J.~Li, S.~Yang, M.~Ji, C.~Yuan, G.~Yang, S.~Yi, and X.~Wang.
\newblock Mobile-television: Predictive motion priors for humanoid whole-body control.
\newblock \emph{arXiv preprint arXiv:2412.07773}, 2024.
\bibitem[Ma et~al.(2022)Ma, Farshidian, Miki, Lee, and Hutter]{ma2022combining}
Y.~Ma, F.~Farshidian, T.~Miki, J.~Lee, and M.~Hutter.
\newblock Combining learning-based locomotion policy with model-based manipulation for legged mobile manipulators.
\newblock \emph{IEEE Robotics and Automation Letters}, 7\penalty0 (2):\penalty0 2377--2384, 2022.
\bibitem[Ewen et~al.(2021)Ewen, Sleiman, Chen, Lu, Hutter, and Vasudevan]{ewen2021generating}
P.~Ewen, J.-P. Sleiman, Y.~Chen, W.-C. Lu, M.~Hutter, and R.~Vasudevan.
\newblock Generating continuous motion and force plans in real-time for legged mobile manipulation.
\newblock In \emph{2021 IEEE international conference on robotics and automation (ICRA)}, pages 4933--4939. IEEE, 2021.
\bibitem[Yokoyama et~al.(2023)Yokoyama, Clegg, Truong, Undersander, Yang, Arnaud, Ha, Batra, and Rai]{yokoyama2023asc}
N.~Yokoyama, A.~Clegg, J.~Truong, E.~Undersander, T.-Y. Yang, S.~Arnaud, S.~Ha, D.~Batra, and A.~Rai.
\newblock Asc: Adaptive skill coordination for robotic mobile manipulation.
\newblock \emph{IEEE Robotics and Automation Letters}, 9\penalty0 (1):\penalty0 779--786, 2023.
\bibitem[Ferrolho et~al.(2023)Ferrolho, Ivan, Merkt, Havoutis, and Vijayakumar]{ferrolho2023roloma}
H.~Ferrolho, V.~Ivan, W.~Merkt, I.~Havoutis, and S.~Vijayakumar.
\newblock Roloma: Robust loco-manipulation for quadruped robots with arms.
\newblock \emph{Autonomous Robots}, 47\penalty0 (8):\penalty0 1463--1481, 2023.
\bibitem[Arcari et~al.(2023)Arcari, Minniti, Scampicchio, Carron, Farshidian, Hutter, and Zeilinger]{arcari2023bayesian}
E.~Arcari, M.~V. Minniti, A.~Scampicchio, A.~Carron, F.~Farshidian, M.~Hutter, and M.~N. Zeilinger.
\newblock Bayesian multi-task learning mpc for robotic mobile manipulation.
\newblock \emph{IEEE Robotics and Automation Letters}, 8\penalty0 (6):\penalty0 3222--3229, 2023.
\bibitem[Zhang et~al.(2024)Zhang, Gireesh, Wang, Fang, Xu, Chen, Dai, and Wang]{zhang2024gamma}
J.~Zhang, N.~Gireesh, J.~Wang, X.~Fang, C.~Xu, W.~Chen, L.~Dai, and H.~Wang.
\newblock Gamma: Graspability-aware mobile manipulation policy learning based on online grasping pose fusion.
\newblock In \emph{2024 IEEE International Conference on Robotics and Automation (ICRA)}, pages 1399--1405. IEEE, 2024.
\bibitem[Jeon et~al.(2023)Jeon, Jung, Choi, Kim, and Hwangbo]{jeon2023learning}
S.~Jeon, M.~Jung, S.~Choi, B.~Kim, and J.~Hwangbo.
\newblock Learning whole-body manipulation for quadrupedal robot.
\newblock \emph{IEEE Robotics and Automation Letters}, 9\penalty0 (1):\penalty0 699--706, 2023.
\bibitem[Ha et~al.(2024)Ha, Gao, Fu, Tan, and Song]{ha2024umi}
H.~Ha, Y.~Gao, Z.~Fu, J.~Tan, and S.~Song.
\newblock {UMI} on legs: Making manipulation policies mobile with manipulation-centric whole-body controllers.
\newblock In \emph{Proceedings of the 2024 Conference on Robot Learning}, 2024.
\bibitem[Liu et~al.(2024)Liu, Chen, Cheng, Ji, Qiu, Yang, and Wang]{liu2024visual}
M.~Liu, Z.~Chen, X.~Cheng, Y.~Ji, R.-Z. Qiu, R.~Yang, and X.~Wang.
\newblock Visual whole-body control for legged loco-manipulation.
\newblock In \emph{8th Annual Conference on Robot Learning}, 2024.
\bibitem[Sleiman et~al.(2024)Sleiman, Mittal, and Hutter]{sleiman2024guided}
J.-P. Sleiman, M.~Mittal, and M.~Hutter.
\newblock Guided reinforcement learning for robust multi-contact loco-manipulation.
\newblock In \emph{8th Annual Conference on Robot Learning (CoRL 2024)}, 2024.
\bibitem[Pan et~al.(2024)Pan, Ben, Yuan, Jiang, Ji, Pang, Liu, and Xu]{pan2024roboduet}
G.~Pan, Q.~Ben, Z.~Yuan, G.~Jiang, Y.~Ji, J.~Pang, H.~Liu, and H.~Xu.
\newblock Roboduet: A framework affording mobile-manipulation and crossembodiment.
\newblock \emph{arXiv preprint arXiv:2403.17367}, 6, 2024.
\bibitem[Fuchs and Kelso(1994)]{fuchs1994theoretical}
A.~Fuchs and J.~Kelso.
\newblock A theoretical note on models of interlimb coordination.
\newblock \emph{Journal of Experimental Psychology: Human Perception and Performance}, 20\penalty0 (5):\penalty0 1088, 1994.
\bibitem[Wannier et~al.(2001)Wannier, Bastiaanse, Colombo, and Dietz]{wannier2001arm}
T.~Wannier, C.~Bastiaanse, G.~Colombo, and V.~Dietz.
\newblock Arm to leg coordination in humans during walking, creeping and swimming activities.
\newblock \emph{Experimental brain research}, 141:\penalty0 375--379, 2001.
\bibitem[Donker et~al.(2001)Donker, Beek, Wagenaar, and Mulder]{donker2001coordination}
S.~F. Donker, P.~J. Beek, R.~Wagenaar, and T.~Mulder.
\newblock Coordination between arm and leg movements during locomotion.
\newblock \emph{Journal of motor behavior}, 33\penalty0 (1):\penalty0 86--102, 2001.
\bibitem[Aoi et~al.(2017)Aoi, Manoonpong, Ambe, Matsuno, and W{\"o}rg{\"o}tter]{aoi2017adaptive}
S.~Aoi, P.~Manoonpong, Y.~Ambe, F.~Matsuno, and F.~W{\"o}rg{\"o}tter.
\newblock Adaptive control strategies for interlimb coordination in legged robots: a review.
\newblock \emph{Frontiers in neurorobotics}, 11:\penalty0 39, 2017.
\bibitem[Tarn et~al.(1986)Tarn, Bejczy, and Yun]{tarn1986coordinated}
T.~Tarn, A.~Bejczy, and X.~Yun.
\newblock Coordinated control of two robot arms.
\newblock In \emph{Proceedings. 1986 IEEE International Conference on Robotics and Automation}, volume~3, pages 1193--1202. IEEE, 1986.
\bibitem[Lin et~al.(2024)Lin, Yin, Qi, Abbeel, and Malik]{lin2024twisting}
T.~Lin, Z.-H. Yin, H.~Qi, P.~Abbeel, and J.~Malik.
\newblock Twisting lids off with two hands.
\newblock In \emph{8th Annual Conference on Robot Learning}, 2024.
\bibitem[Wolfslag et~al.(2020)Wolfslag, McGreavy, Xin, Tiseo, Vijayakumar, and Li]{wolfslag2020optimisation}
W.~J. Wolfslag, C.~McGreavy, G.~Xin, C.~Tiseo, S.~Vijayakumar, and Z.~Li.
\newblock Optimisation of body-ground contact for augmenting the whole-body loco-manipulation of quadruped robots.
\newblock In \emph{2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)}, pages 3694--3701. IEEE, 2020.
\bibitem[Lin et~al.(2024)Lin, Liu, Yang, Niu, Yu, Zhang, Tan, Boots, and Zhao]{lin2024locoman}
C.~Lin, X.~Liu, Y.~Yang, Y.~Niu, W.~Yu, T.~Zhang, J.~Tan, B.~Boots, and D.~Zhao.
\newblock Locoman: Advancing versatile quadrupedal dexterity with lightweight loco-manipulators.
\newblock In \emph{2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)}, pages 6877--6884. IEEE, 2024.
\bibitem[Arm et~al.(2024)Arm, Mittal, Kolvenbach, and Hutter]{arm2024pedipulate}
P.~Arm, M.~Mittal, H.~Kolvenbach, and M.~Hutter.
\newblock Pedipulate: Enabling manipulation skills using a quadruped robot’s leg.
\newblock In \emph{2024 IEEE International Conference on Robotics and Automation (ICRA)}, pages 5717--5723. IEEE, 2024.
\bibitem[He et~al.(2024)He, Lei, Ze, Sreenath, Li, and Xu]{he2024learning}
Z.~He, K.~Lei, Y.~Ze, K.~Sreenath, Z.~Li, and H.~Xu.
\newblock Learning visual quadrupedal loco-manipulation from demonstrations.
\newblock In \emph{2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)}, pages 9102--9109. IEEE, 2024.
\bibitem[Cheng et~al.(2023)Cheng, Kumar, and Pathak]{cheng2023legs}
X.~Cheng, A.~Kumar, and D.~Pathak.
\newblock Legs as manipulator: Pushing quadrupedal agility beyond locomotion.
\newblock In \emph{2023 IEEE International Conference on Robotics and Automation (ICRA)}, pages 5106--5112. IEEE, 2023.
\bibitem[Zakka(2024)]{Zakka_Mink_Python_inverse_2024}
K.~Zakka.
\newblock Mink: Python inverse kinematics based on mujoco, July 2024.
\newblock URL \url{https://github.com/kevinzakka/mink}.
\newblock License: Apache-2.0.
\bibitem[Mittal et~al.(2023)Mittal, Yu, Yu, Liu, Rudin, Hoeller, Yuan, Singh, Guo, Mazhar, Mandlekar, Babich, State, Hutter, and Garg]{mittal2023orbit}
M.~Mittal, C.~Yu, Q.~Yu, J.~Liu, N.~Rudin, D.~Hoeller, J.~L. Yuan, R.~Singh, Y.~Guo, H.~Mazhar, A.~Mandlekar, B.~Babich, G.~State, M.~Hutter, and A.~Garg.
\newblock Orbit: A unified simulation framework for interactive robot learning environments.
\newblock \emph{IEEE Robotics and Automation Letters}, 8\penalty0 (6):\penalty0 3740--3747, 2023.
\newblock \doi{10.1109/LRA.2023.3270034}.
\bibitem[Miller et~al.(2025)Miller, Yu, Brauckmann, and Farshidian]{miller2025high}
A.~Miller, F.~Yu, M.~Brauckmann, and F.~Farshidian.
\newblock High-performance reinforcement learning on spot: Optimizing simulation parameters with distributional measures.
\newblock \emph{arXiv preprint arXiv:2403.16967}, 2025.
\bibitem[Siekmann et~al.(2021)Siekmann, Godse, Fern, and Hurst]{siekmann2021gait}
J.~Siekmann, Y.~Godse, A.~Fern, and J.~Hurst.
\newblock Sim-to-real learning of all common bipedal gaits via periodic reward composition.
\newblock In \emph{2021 IEEE International Conference on Robotics and Automation (ICRA)}, pages 7309--7315, 2021.
\newblock \doi{10.1109/ICRA48506.2021.9561814}.
\bibitem[Miller et~al.(2011)Miller, Fritz, Darrell, and Abbeel]{miller2011parametrized}
S.~Miller, M.~Fritz, T.~Darrell, and P.~Abbeel.
\newblock Parametrized shape models for clothing.
\newblock In \emph{2011 IEEE International Conference on Robotics and Automation}, pages 4861--4868. IEEE, 2011.
\bibitem[Nomura and Shibata(2024)]{nomura2024cmaes}
M.~Nomura and M.~Shibata.
\newblock cmaes : A simple yet practical python library for cma-es.
\newblock \emph{arXiv preprint arXiv:2402.01373}, 2024.
\bibitem[OpenAI(2024)]{openai2024gpt4ocard}
OpenAI.
\newblock Gpt-4o system card, 2024.
\newblock URL \url{https://arxiv.org/abs/2410.21276}.
\bibitem[{Boston Dynamics}({\natexlab{a}})]{bostondynamics_spot}
{Boston Dynamics}.
\newblock Spot.
\newblock \url{https://bostondynamics.com/products/spot/}, {\natexlab{a}}.
\bibitem[{Boston Dynamics}({\natexlab{b}})]{bostondynamics_api}
{Boston Dynamics}.
\newblock Spot sdk.
\newblock \url{https://dev.bostondynamics.com/}, {\natexlab{b}}.
\bibitem[developers(2021)]{onnxruntime}
O.~R. developers.
\newblock Onnx runtime.
\newblock \url{https://onnxruntime.ai/}, 2021.
\bibitem[Rudin et~al.(2022)Rudin, Hoeller, Reist, and Hutter]{rudin2022learning}
N.~Rudin, D.~Hoeller, P.~Reist, and M.~Hutter.
\newblock Learning to walk in minutes using massively parallel deep reinforcement learning.
\newblock In \emph{Conference on Robot Learning}, pages 91--100. PMLR, 2022.
\bibitem[Ravi et~al.(2024)Ravi, Gabeur, Hu, Hu, Ryali, Ma, Khedr, R{\"a}dle, Rolland, Gustafson, Mintun, Pan, Alwala, Carion, Wu, Girshick, Doll{\'a}r, and Feichtenhofer]{ravi2024sam2}
N.~Ravi, V.~Gabeur, Y.-T. Hu, R.~Hu, C.~Ryali, T.~Ma, H.~Khedr, R.~R{\"a}dle, C.~Rolland, L.~Gustafson, E.~Mintun, J.~Pan, K.~V. Alwala, N.~Carion, C.-Y. Wu, R.~Girshick, P.~Doll{\'a}r, and C.~Feichtenhofer.
\newblock Sam 2: Segment anything in images and videos.
\newblock \emph{arXiv preprint arXiv:2408.00714}, 2024.
\newblock URL \url{https://arxiv.org/abs/2408.00714}.
\end{thebibliography}