MotionScape: A Large-Scale Real-World Highly Dynamic UAV Video Dataset for World Models
Extracted source text · arXiv 2604.07991
%%
%% This is file `sample-sigconf.tex',
%% generated with the docstrip utility.
%%
%% The original source files were:
%%
%% samples.dtx (with options: `all,proceedings,bibtex,sigconf')
%%
%% IMPORTANT NOTICE:
%%
%% For the copyright see the source file.
%%
%% Any modified versions of this file must be renamed
%% with new filenames distinct from sample-sigconf.tex.
%%
%% For distribution of the original source see the terms
%% for copying and modification in the file samples.dtx.
%%
%% This generated file may be distributed as long as the
%% original source files, as listed above, are part of the
%% same distribution. (The sources need not necessarily be
%% in the same archive or directory.)
%%
%%
%% Commands for TeXCount
%TC:macro \cite [option:text,text]
%TC:macro \citep [option:text,text]
%TC:macro \citet [option:text,text]
%TC:envir table 0 1
%TC:envir table* 0 1
%TC:envir tabular [ignore] word
%TC:envir displaymath 0 word
%TC:envir math 0 word
%TC:envir comment 0 0
%%
%% The first command in your LaTeX source must be the \documentclass
%% command.
%%
%% For submission and review of your manuscript please change the
%% command to \documentclass[manuscript, screen, review]{acmart}.
%%
%% When submitting camera ready or to TAPS, please change the command
%% to \documentclass[sigconf]{acmart} or whichever template is required
%% for your publication.
%%
%%
% \documentclass[sigconf]{acmart}
\documentclass[sigconf]{acmart}
%%
%% \BibTeX command to typeset BibTeX logo in the docs
\AtBeginDocument{%
\providecommand\BibTeX{{%
Bib\TeX}}}
%% Rights management information. This information is sent to you
%% when you complete the rights form. These commands have SAMPLE
%% values in them; it is your responsibility as an author to replace
%% the commands and values with those provided to you when you
%% complete the rights form.
\setcopyright{acmlicensed}
\copyrightyear{2018}
\acmYear{2018}
\acmDOI{XXXXXXX.XXXXXXX}
%% These commands are for a PROCEEDINGS abstract or paper.
% \acmConference[Conference acronym 'XX]{Make sure to enter the correct
% conference title from your rights confirmation email}{June 03--05,
% 2018}{Woodstock, NY}
%%
%% Uncomment \acmBooktitle if the title of the proceedings is different
%% from ``Proceedings of ...''!
%%
%%\acmBooktitle{Woodstock '18: ACM Symposium on Neural Gaze Detection,
%% June 03--05, 2018, Woodstock, NY}
\acmISBN{978-1-4503-XXXX-X/2018/06}
%%
%% Submission ID.
%% Use this when submitting an article to a sponsored event. You'll
%% receive a unique submission ID from the organizers
%% of the event, and this ID should be used as the parameter to this command.
%%\acmSubmissionID{123-A56-BU3}
%%
%% For managing citations, it is recommended to use bibliography
%% files in BibTeX format.
%%
%% You can then either use BibTeX with the ACM-Reference-Format style,
%% or BibLaTeX with the acmnumeric or acmauthoryear sytles, that include
%% support for advanced citation of software artefact from the
%% biblatex-software package, also separately available on CTAN.
%%
%% Look at the sample-*-biblatex.tex files for templates showcasing
%% the biblatex styles.
%%
%%
%% The majority of ACM publications use numbered citations and
%% references. The command \citestyle{authoryear} switches to the
%% "author year" style.
%%
%% If you are preparing content for an event
%% sponsored by ACM SIGGRAPH, you must use the "author year" style of
%% citations and references.
%% Uncommenting
%% the next command will enable that style.
%%\citestyle{acmauthoryear}
\renewcommand\footnotetextcopyrightpermission[1]{}
\settopmatter{printacmref=false}
%%
%% end of the preamble, start of the body of the document source.
\begin{document}
%%
%% The "title" command has an optional parameter,
%% allowing the author to define a "short title" to be used in page headers.
\title{MotionScape: A Large-Scale Real-World Highly Dynamic UAV Video Dataset for World Models}
%%
%% The "author" command and its associated commands are used to define
%% the authors and their affiliations.
%% Of note is the shared affiliation of the first two authors, and the
%% "authornote" and "authornotemark" commands
%% used to denote shared contribution to the research.
\author{Zile Guo}
\email{guozile25@mails.ucas.ac.cn}
\affiliation{%
\institution{Aerospace Information Research Institute, Chinese Academy of Sciences}
\state{Beijing}
\country{China}
}
\author{Zhan Chen}
\email{chenzhan21@mails.ucas.ac.cn}
\affiliation{%
\institution{Aerospace Information Research Institute, Chinese Academy of Sciences}
\state{Beijing}
\country{China}
}
\author{Enze Zhu}
\email{zhuenze23@mails.ucas.ac.cn}
\affiliation{%
\institution{Aerospace Information Research Institute, Chinese Academy of Sciences}
\state{Beijing}
\country{China}
}
\author{Kan Wei}
\email{weikan24@mails.ucas.ac.cn}
\affiliation{%
\institution{Aerospace Information Research Institute, Chinese Academy of Sciences}
\state{Beijing}
\country{China}
}
\author{Yongkang Zou}
\email{zouyk2022@mails.jlu.edu.cn}
\affiliation{%
\institution{Jilin University}
\state{Jilin}
\country{China}}
\author{Xiaoxuan Liu}
\email{liuxiaoxuan@aircas.ac.cn}
\affiliation{%
\institution{Aerospace Information Research Institute, Chinese Academy of Sciences}
\state{Beijing}
\country{China}}
\author{Lei Wang}
\email{wanglei002931@aircas.ac.cn}
\affiliation{%
\institution{Aerospace Information Research Institute, Chinese Academy of Sciences}
\state{Beijing}
\country{China}}
%%
%% By default, the full list of authors will be used in the page
%% headers. Often, this list is too long, and will overlap
%% other information printed in the page headers. This command allows
%% the author to define a more concise list
%% of authors' names for this purpose.
% \renewcommand{\shortauthors}{Trovato et al.}
%%
%% The abstract is a short summary of the work to be presented in the
%% article.
\begin{abstract}
Recent advances in world models have demonstrated strong capabilities in simulating physical reality, making them an increasingly important foundation for embodied intelligence. For UAV agents in particular, accurate prediction of complex 3D dynamics is essential for autonomous navigation and robust decision-making in unconstrained environments. However, under the highly dynamic camera trajectories typical of UAV views, existing world models often struggle to maintain spatiotemporal physical consistency. A key reason lies in the distribution bias of current training data: most existing datasets exhibit restricted 2.5D motion patterns, such as ground-constrained autonomous driving scenes or relatively smooth human-centric egocentric videos, and therefore lack realistic high-dynamic 6-DoF UAV motion priors. To address this gap, we present MotionScape, a large-scale real-world UAV-view video dataset with highly dynamic motion for world modeling. MotionScape contains over 30 hours of 4K UAV-view videos, totaling more than 4.5M frames. This novel dataset features semantically and geometrically aligned training samples, where diverse real-world UAV videos are tightly coupled with accurate 6-DoF camera trajectories and fine-grained natural language descriptions. To build the dataset, we develop an automated multi-stage processing pipeline that integrates CLIP-based relevance filtering, temporal segmentation, robust visual SLAM for trajectory recovery, and large-language-model-driven semantic annotation. By explicitly disentangling complex ego-motion from scene dynamics, MotionScape provides a data foundation with strong geometric constraints for world modeling. Extensive experiments show that incorporating such semantically and geometrically aligned annotations effectively improves the ability of existing world models to simulate complex 3D dynamics and handle large viewpoint shifts, thereby benefiting decision-making and planning for UAV agents in complex environments. The dataset is publicly available at https://github.com/Thelegendzz/MotionScape
\end{abstract}
%%
%% The code below is generated by the tool at http://dl.acm.org/ccs.cfm.
%% Please copy and paste the code instead of the example below.
%%
\begin{CCSXML}
<ccs2012>
<concept>
<concept_id>10010147.10010178.10010224.10010226.10010239</concept_id>
<concept_desc>Computing methodologies~3D imaging</concept_desc>
<concept_significance>500</concept_significance>
</concept>
<concept>
<concept_id>10010147.10010178.10010224.10010225.10010227</concept_id>
<concept_desc>Computing methodologies~Scene understanding</concept_desc>
<concept_significance>300</concept_significance>
</concept>
<concept>
<concept_id>10010147.10010178.10010224.10010225.10010233</concept_id>
<concept_desc>Computing methodologies~Vision for robotics</concept_desc>
<concept_significance>100</concept_significance>
</concept>
</ccs2012>
\end{CCSXML}
\ccsdesc[500]{Computing methodologies~3D imaging}
\ccsdesc[300]{Computing methodologies~Scene understanding}
\ccsdesc[100]{Computing methodologies~Vision for robotics}
%%
%% Keywords. The author(s) should pick words that accurately describe
%% the work being presented. Separate the keywords with commas.
\keywords{World models, UAV video dataset, 6-DoF camera motion, high-dynamic scenes}
%% A "teaser" image appears between the author and affiliation
%% information and the body of the document, and typically spans the
%% page.
\begin{teaserfigure}
\includegraphics[width=\textwidth]{fig/problem.pdf}
\caption{Example output sequence of Cosmos 2.5-2B for video continuation in a highly dynamic UAV scenario. The zoomed-in regions reveal detail loss and local geometric inconsistency across consecutive generated frames.}
\label{fig:teaser}
\end{teaserfigure}
% \received{20 February 2007}
% \received[revised]{12 March 2009}
% \received[accepted]{5 June 2009}
%%
%% This command processes the author and affiliation and title
%% information and builds the first part of the formatted document.
\maketitle
\fancyhead{}
\fancyfoot{}
\renewcommand{\headrulewidth}{0pt}
\renewcommand{\footrulewidth}{0pt}
\section{Introduction}
Recent advances in world models~\cite{ha2018worldmodels} have sparked a paradigm shift in generative AI, demonstrating unprecedented capabilities in simulating complex physical dynamics and rendering highly realistic video sequences. Pioneering systems such as Sora~\cite{liu2024sorareviewbackgroundtechnology}, Cosmos~\cite{nvidia2026worldsimulationvideofoundation}, and Gen-3 Alpha have significantly advanced text-to-video and image-to-video generation by scaling diffusion and Transformer-based architectures~\cite{peebles2023dit}. These models aim to internalize physical laws, enabling the simulation of real-world interactions ranging from fluid dynamics to rigid-body motion.
However, despite these advances, we observe that current world models still struggle to faithfully preserve fine-grained geometric structure under highly dynamic camera trajectories, particularly in aerial scenarios. As illustrated in Fig~\ref{fig:teaser}, even advanced models such as Cosmos 2.5-2B can roughly maintain the global scene layout, but often fail to preserve locally consistent geometry and texture details under rapid viewpoint changes~\cite{sim2024evaluating}. These failures suggest that purely semantic conditioning is insufficient for modeling complex spatiotemporal dynamics, highlighting a critical gap in current training data.
We argue that this limitation stems from the lack of datasets that capture physically grounded camera motion in open-world environments. Existing datasets predominantly exhibit quasi-2D motion patterns, such as ground-constrained trajectories in autonomous driving or limited egocentric viewpoints in human-centric data. Although robotic platforms~\cite{padalkar2023openx} provide 6-DoF interactions, their motion remains confined to restricted workspaces, and thus fails to simultaneously satisfy both highly dynamic 6-DoF motion and large-scale open-world coverage.
In contrast, UAVs are uniquely positioned to fulfill both requirements. They enable high-speed, unconstrained 6-DoF motion across diverse real-world environments, covering complex large-scale 3D scenes~\cite{loquercio2021learning}. This allows UAVs to capture highly dynamic motion patterns, including rapid dives, sharp rotations, and large viewpoint changes, which provide supervision over both complex physical motion and diverse real-world scenes. Such data is crucial for learning world models with true 3D reasoning capabilities and generalizable spatiotemporal dynamics.
\begin{figure}
\includegraphics[width=\linewidth]{fig/distribution.pdf}
\caption{Distribution of video resolutions and environmental conditions in our dataset.}
\label{fig:distribution}
\end{figure}
Our contributions are summarized as follows:
• An empirical analysis for high-dynamic UAV world modeling.
Through motion statistics and generation experiments, we show that MotionScape exhibits substantially stronger dynamicity than existing real-world UAV datasets while approaching the motion intensity of highly challenging synthetic data. We further show that even advanced video generation models remain limited in exploiting auxiliary conditions under highly dynamic UAV video continuation.
• A scalable pipeline for multimodal data construction.
We develop an automated multi-stage pipeline that transforms raw Internet UAV videos into structured training data for world models. The pipeline integrates semantic filtering, temporal segmentation, caption generation, and robust pose recovery, producing video segments paired with natural language descriptions and 6-DoF camera trajectories.
• An empirical analysis for high-dynamic UAV world modeling.
Through motion statistics and generation experiments, we show that MotionScape exhibits substantially stronger dynamicity than existing real-world UAV datasets while approaching the motion intensity of highly challenging synthetic data. We further show that even advanced video generation models remain limited in exploiting auxiliary conditions under highly dynamic UAV video continuation.
\section{Related Work}
\subsection{Video Generation Datasets and World Models}
From public datasets built from open-domain Internet videos, such as WebVid-10M~\cite{chen2024videocrafter2overcomingdatalimitations}, Panda-70M~\cite{chen2024panda70mcaptioning70mvideos}, and OpenVid-1M~\cite{nan2025openvid1mlargescalehighqualitydataset}, to the large-scale private video corpora used by recent frontier world models, massive video data has significantly improved training scale, visual fidelity, and semantic alignment. Meanwhile, domain-specific datasets such as Kinetics~\cite{kay2017kineticshumanactionvideo}, Ego4D~\cite{grauman2022ego4dworld3000hours}, nuScenes~\cite{caesar2020nuscenesmultimodaldatasetautonomous}, and Waymo~\cite{sun2020scalabilityperceptionautonomousdriving} have also played an important role in areas including autonomous driving and embodied perception. However, public video datasets are typically sourced from edited Internet videos, and therefore lack explicit pose annotations and consistent 3D geometric constraints. By contrast, although embodied and driving datasets provide physically grounded trajectories, their camera motions are often restricted by human behavior patterns or road topology, leading to a highly constrained quasi-2D motion distribution. Consequently, current training data for world models still lacks explicit pose supervision and sufficient coverage of highly dynamic free six-degree-of-freedom (6-DoF) camera motion, limiting their ability to maintain 3D consistency under drastic viewpoint changes and large spatial displacements.
\subsection{UAV and Aerial Datasets}
Existing UAV and aerial-view datasets are primarily designed for detection, tracking, segmentation, action recognition, or visual localization, rather than explicitly for video generation or world model training. As a result, although these datasets each offer certain advantages in resolution, scene realism, or local geometric information, they still exhibit clear gaps in several properties required by world models, including long-horizon temporal continuity, 6-DoF camera motion, highly dynamic 3D trajectories, and semantic alignment.
Specifically, datasets such as VisDrone~\cite{zhu2018vision} and UAVid~\cite{lyu2020uavid} provide real-world high-resolution aerial videos, but usually lack explicit camera trajectories and geometric supervision, making it difficult to disentangle ego-motion from scene dynamics. Datasets such as Zurich Urban MAV~\cite{majdik2017zurich} and ACID~\cite{liu2021infinite} begin to provide ground-truth or recovered camera poses, but are often limited in scale, scene diversity, or motion range. Meanwhile, Mid-Air~\cite{fonder2019mid} and Blackbird~\cite{antonini2018blackbird} offer stronger geometric supervision or more dynamic motion, yet the former is synthetic and the latter is mainly collected in controlled environments, making both insufficient for covering complex UAV maneuvers in open real-world scenes. In addition, most existing UAV datasets provide only task-oriented annotations such as bounding boxes, segmentation masks, or action labels, while lacking semantic descriptions that are directly useful for text-conditioned generation.
Overall, existing UAV datasets remain fragmented across resolution, scale, dynamics, geometric supervision, and semantic annotation, and therefore still fall short of providing a unified data foundation for high-dynamic world model training and camera-controllable generation~\cite{wang2024motionctrl, he2024cameractrl}.
\begin{table*}[t]
\caption{Comparison of UAV datasets. Our dataset uniquely combines high-resolution, large-scale coverage, highly dynamic 6-DoF motion, real-world diversity, and semantic annotations. GT is short for Ground Truth. W\&E stands for weather and environment.}
\centering
\small
\resizebox{\textwidth}{!}{
\begin{tabular}{lccccc}
\toprule
\textbf{Dataset} & \textbf{Resolution} & \textbf{Scale} & \textbf{6-DoF Pose} & \textbf{Real/Synthetic} & \textbf{Semantic Annotation} \\
\midrule
VisDrone~\cite{zhu2018vision} & up to 3840$\times$2160 & 400 clips / 265,228 frames & None & Real & None \\
UAVid~\cite{lyu2020uavid} & 3840$\times$2160 & 42 sequences & None & Real & None \\
ACID~\cite{liu2021infinite} & Variable & 2.1M frames & Estimated (SfM) & Real & None \\
Blackbird~\cite{antonini2018blackbird} & 1024×768 & 163 flights / $>$10 hours & GT & Mixed & None \\
Zurich Urban MAV~\cite{majdik2017zurich} & 1920$\times$1080 & 2km trajectory & GT & Real & None \\
Mid-Air~\cite{fonder2019mid} & 1024$\times$1024 & 54 trajectories / 79 min ($>$420K frames) & GT & Synthetic & None \\
\midrule
\textbf{MotionScape(Ours)} & \textbf{3840$\times$2160} & \textbf{4.5M frames\/> 30 hours} & \textbf{Estimated (SLAM)} & \textbf{Real} & \textbf{Scene Meta(W\&E) + Caption} \\
\bottomrule
\end{tabular}}
\label{tab:uav_dataset}
\end{table*}
\subsection{Camera Trajectory Extraction in the Wild}
Since most existing UAV and aerial-view datasets do not provide 6-DoF camera trajectories that can be directly used for world model training, recovering camera motion from pure visual videos becomes a key step in constructing such data. Existing approaches mainly include classical geometric methods, such as Structure-from-Motion (SfM)~\cite{7780814} and visual SLAM~\cite{campos2021orb}, as well as more recent learning-based visual SLAM methods (e.g., DROID-SLAM~\cite{teed2022droidslamdeepvisualslam}) and NeRF-based joint pose optimization methods(e.g., BARF~\cite{lin2021barf} and NeRF--~\cite{wang2021nerf}).Compared with classical geometric pipelines, learning-based visual SLAM methods are generally more robust to motion blur, rapid viewpoint changes, and long-horizon videos, while NeRF-based methods suggest that usable camera trajectories can be jointly optimized during reconstruction even without highly accurate initial poses.
Although trajectories recovered by these methods are usually less accurate than high-precision ground-truth poses, what matters more for world models is temporally consistent, physically plausible camera motion that aligns with video content, rather than perfect geometric accuracy. Especially in highly dynamic and open-world video scenarios, constructing large-scale datasets with GT poses is often impractical. Therefore, recovering usable 6-DoF camera trajectories from pure videos has become a realistic and scalable technical path for building training data for high-dynamic world models.
\section{MotionScape}
\subsection{Overview and Statistics}
\begin{figure*}
\includegraphics[width=\textwidth]{fig/dataset.pdf}
\caption{Representative samples illustrating the scene and weather diversity of our dataset, including mountain, indoor, waterfall, city, forest, desert, coast, and volcano scenes, as well as overcast, rainy, snowy, and foggy conditions.}
\label{fig:dataset}
\end{figure*}
We introduce MotionScape, a high-dynamic UAV video dataset for world model training. Built upon high-resolution aerial videos, the dataset focuses on modeling complex real-world six-degree-of-freedom (6-DoF) camera motion, aiming to support spatiotemporal modeling and controllable generation in high-degree-of-freedom scenarios. Unlike existing UAV datasets that are primarily designed for detection, tracking, or localization, MotionScape jointly provides high-dynamic videos, recoverable 6-DoF camera trajectories, and fine-grained semantic information aligned with video content, forming a training resource with visual, geometric, and semantic attributes.
In terms of visual quality, the dataset is centered on 4K-resolution videos, preserving rich textures and spatial details in complex aerial scenes. In terms of motion, it contains highly dynamic camera movements with large viewpoint changes, rapid translations, and significant pose variations, helping cover a motion distribution that is largely missing from existing world model training data. In addition to camera trajectory information, each video clip is annotated with fine-grained semantic metadata aligned with its content, including environment type, weather condition, and scene attributes, which supports text-guided generation and joint semantic-geometric modeling.
Regarding scene coverage, MotionScape includes both natural and human-made environments. Natural scenes cover representative aerial landscapes such as mountains, coastlines, and waterfalls, while human-made scenes include urban buildings, roads, and other artificial environments. The dataset also spans diverse weather conditions, including sunny, rainy, and foggy scenes, introducing richer appearance variation and visual challenges. Overall, the dataset provides a realistic, semantically rich, and geometrically grounded data foundation for studying world models under highly dynamic 6-DoF camera motion.
\subsection{Data Curation Pipeline}
The raw candidate videos in our dataset are mainly drawn from manually pre-screened UAV video compilations on YouTube. Based on these candidate videos, we design a three-stage data curation pipeline to progressively remove non-target content and enrich the retained clips with geometric and semantic information, ultimately constructing structured training data for world models.
\subsubsection{CLIP-based Coarse Filtering}
We first employ CLIP~\cite{radford2021learningtransferablevisualmodels} for coarse-grained target-relevance filtering on the candidate videos. Specifically, we use ViT-B/32~\cite{dosovitskiy2020image} with OpenAI pretrained weights to match video frames, sampled at xx fps, against a predefined set of FPV and non-FPV text prompts, and compute for each frame a probability score of belonging to any FPV prompt. This score is defined as the probability that a frame belongs to any FPV prompt under the joint competition of all FPV and non-FPV prompts; a score closer to 1 indicates that the frame is more likely to correspond to target FPV content, whereas a lower score suggests non-FPV content. Based on this score, we filter the sampled frames using a threshold of xx. Since the raw candidate videos have already been manually pre-screened in the previous stage, the goal of this step is no longer broad retrieval of UAV-related videos, but rather the further removal of residual non-target content within the candidate pool, such as black frames, intro/title screens, talking-head segments, and third-person views of drones. Under this setting, non-FPV prompts play the dominant role in filtering, while FPV prompts mainly serve as auxiliary confirmation of whether a clip is consistent with the target aerial content. In total, this stage removes approximately xx\% of the sampled video frames (xx frames in total).
\subsubsection{Temporal Segmentation with TransNetV2}
After coarse-grained relevance filtering, we further apply TransNetV2~\cite{souček2020transnetv2effectivedeep} to candidate videos to detect shot boundaries and identify temporal discontinuities. Specifically, we use the publicly available pretrained TransNetV2 model to split long videos into temporally coherent clips. Since clips shorter than 2 seconds usually contain limited viewpoint changes and motion information, offering insufficient continuous trajectory variation for downstream processing, we discard such clips and retain only those with sufficient duration and temporal coherence.
\subsubsection{Semantic Annotation}
After shot boundary detection, we further generate semantic metadata for the retained clips to support downstream text-guided generation and joint semantic-geometric modeling. We employ GPT-4o mini~\cite{openai2024gpt4ocard} for automatic semantic annotation of video clips~\cite{chen2024panda70mcaptioning70mvideos, wang2023internvid}. Since a single frame often fails to fully capture scene dynamics and contextual information, we sample each video at 1 FPS and use a 5-second sliding temporal window as the annotation unit, providing the model with sufficient temporal context to infer scene attributes and overall motion trends.
During annotation, we first ask the model to determine whether the current clip corresponds to valid drone flight footage, which helps further filter residual non-target content such as talking-head segments, intro screens, or other non-flight clips. For clips that pass this validity check, we then generate semantic annotations aligned with the video content, including weather conditions, environment descriptions, and natural-language captions. The weather label mainly covers common categories such as sunny, overcast, rainy, foggy, and night scenes. The environment label is used to summarize the scene attribute. Since real-world environments are often compositionally complex, enforcing a predefined label space may lead to the loss of semantic information; we therefore represent the environment using a short text description of no more than five words. In addition, the model generates a fine-grained 2–3 sentence caption describing the scene, lighting conditions, and the overall viewpoint changes and coarse motion patterns of the clip.
\begin{figure*}
\includegraphics[width=\textwidth]{fig/pipeline.pdf}
\caption{Distribution of video resolutions and environmental conditions in our dataset.}
\label{fig:pipeline}
\end{figure*}
\subsubsection{Pose Recovery}
To provide explicit camera-motion supervision for world model training, we further recover 6-DoF camera trajectories for the retained clips using DROID-SLAM~\cite{teed2022droidslamdeepvisualslam}. DROID-SLAM is a representative deep visual SLAM system that jointly optimizes camera poses and scene geometry, and has shown strong robustness under motion blur, rapid viewpoint changes, and long-horizon videos, making it suitable for high-dynamic UAV footage.
In practice, we run DROID-SLAM on each temporally coherent clip and use the recovered trajectories as geometric metadata. It is important to note that, under the monocular setting, DROID-SLAM mainly recovers relative camera trajectories, whose global scale is generally not uniquely determined~\cite{hartley2003multiple}. However, for world models, what matters more is the temporal consistency of camera motion, the relative displacement pattern, and the associated viewpoint changes, rather than perfectly accurate geometry under an absolute global scale. Therefore, although such trajectories cannot replace high-precision ground-truth poses, they still provide effective supervision for learning the correspondence between camera motion and visual variation. Based on this principle, we further discard clips with failed or severely unstable tracking results, retaining only clips with stable relative trajectories for subsequent data construction.
\section{Experiments}
In this section, we conduct experiments from two perspectives. First, we evaluate the dynamic characteristics of the constructed dataset through optical-flow-based statistics and compare them with existing UAV datasets, in order to analyze its motion distribution and high-dynamic properties. Second, we investigate the limitations of existing generation models in highly dynamic UAV video continuation. Specifically, without any additional post-training, we introduce caption and pose information as auxiliary conditions into the generation process and compare the outputs under different control settings, so as to analyze the practical role of multimodal signals in controllable video generation.
\subsection{Implementation Details}
\subsubsection{Optical Flow Analysis}
In this part, we randomly sample 50 video clips from the full dataset as the evaluation subset. For consistent motion statistics computation, all sampled clips are uniformly processed at 5 FPS and downsampled to a resolution of 854×480. For each video clip, we first estimate the optical flow field between consecutive sampled frames and compute the flow magnitude at each pixel as
\begin{equation}
m_t(x,y)=\sqrt{u_t(x,y)^2+v_t(x,y)^2}
\end{equation}
where $u_t(x,y)$ and $v_t(x,y)$ denote the horizontal and vertical flow components at pixel location $(x,y)$ for the $t$-th pair of consecutive frames, respectively. We then spatially average the flow magnitude for each frame pair to obtain its mean motion intensity, and further average the resulting values over all consecutive frame pairs in a video to obtain a video-level mean flow magnitude. This two-stage averaging scheme preserves the overall motion trend while reducing the influence of locally extreme motion or short-term fluctuations, thereby providing a more stable characterization of the overall dynamicity of a video clip. A larger mean flow magnitude generally indicates stronger motion variation and higher dynamic difficulty.
\subsubsection{Generation Evaluation}
We use NVIDIA Cosmos 2.5-2B as the baseline model for generation evaluation. As a representative world foundation model, it supports multiple inference modes, including Text2world, Image2world, and Video2world generation, enabling unified text-guided generation, image-conditioned generation, and video continuation within the same framework. Its strong predictive generation capability and support for high-resolution video synthesis make it a suitable baseline for evaluating performance on highly dynamic UAV videos.
In this work, we adopt its video2world inference mode for video continuation.We design three conditioning strategies to compare generation performance under different levels of control. First, the model directly extends the input video without any additional control signal, which serves as the baseline setting. Second, captions generated by GPT-4o mini in the data pipeline are introduced as auxiliary textual conditions for video continuation. Finally, relative pose information is further incorporated to explicitly guide the camera pose of each generated frame. Through these settings, we examine whether the auxiliary information constructed in our dataset can improve the generation performance of Cosmos 2.5-2B in highly dynamic UAV scenarios.
\subsubsection{Evaluation Metrics}
For evaluation, we assess the generated results from three aspects: visual fidelity, temporal consistency, and overall video quality. Specifically, PSNR and SSIM are used to measure pixel-level and structural similarity between the generated videos and the reference videos, while LPIPS is adopted to evaluate perceptual visual discrepancy. To further assess temporal coherence, we introduce Warping Error, which measures whether the motion relationship between consecutive frames is consistent and whether the generated video remains stable over time. In addition, we employ Fréchet Video Distance (FVD) to evaluate the discrepancy between generated and real videos in the distribution of high-level video features, thereby reflecting the overall video quality. Based on these metrics, we compare the effect of different control settings on generation performance in highly dynamic UAV scenes.
\subsubsection{Experimental Setup}
Video generation experiments are conducted on four RTX 6000 Ada GPUs, while all other experiments are performed on a single RTX 6000 Ada GPU. In the generation evaluation, we use the first 200 observed frames as input and assess generation performance on a 77-frame target segment, which consists of the last 2 conditioning frames from the input sequence and 75 future frames to be predicted. No additional post-training is applied in any experiment.
\subsection{Results}
As shown in Table~\ref{tab:flow_magnitude}, MotionScape achieves a mean flow magnitude of 36.459, which is substantially higher than several existing real-world UAV datasets, including VisDrone (12.663), UAVid (9.745), and Zurich Urban MAV (4.416). This result indicates that MotionScape exhibits much stronger motion variation and more pronounced viewpoint changes in real-world UAV videos, thereby providing a more challenging dynamic distribution for world modeling. Notably, although MotionScape is constructed entirely from real-world UAV footage, its mean flow magnitude is only slightly lower than that of the highly dynamic synthetic dataset TartanAir (41.856). This suggests that MotionScape approaches the motion complexity of deliberately designed high-challenge synthetic environments while preserving the realism of real-world UAV videos. Overall, these results demonstrate that MotionScape offers significantly stronger dynamicity than existing real-world UAV datasets and serves as a more challenging and realistic benchmark for high-dynamic world modeling.
As a representative advanced video generation model, Cosmos 2.5-2B provides a strong baseline for evaluating video continuation under challenging UAV scenarios. However,as shown in Table~\ref{tab:baseline}, our experiments show that, under the video2world setting, its responsiveness to auxiliary control signals remains limited in highly dynamic UAV video continuation. Neither textual guidance nor relative pose conditioning leads to clear improvement over the uncontrolled baseline. Although the model generally preserves a roughly continuous motion trend, the generated results still suffer from degraded fine details, blurred local structures, and insufficient spatial texture preservation under highly dynamic camera motion and large viewpoint changes. This suggests that, despite its strong general-purpose generation capability, current models still struggle to effectively exploit semantic or geometric conditions to improve fine-grained fidelity and controllable generation quality under complex 6-DoF motion.
These findings further reveal an important limitation of existing world models and video generation models: current training data and learned priors remain insufficient for highly dynamic real-world UAV scenarios. Existing datasets generally lack realistic high-dynamic UAV motion distributions with complex viewpoint changes, while current models still show limited ability to exploit fine-grained motion priors and explicit geometric constraints. This highlights the importance of building datasets that jointly provide highly dynamic real-world UAV videos, accurate 6-DoF camera trajectories, and structured semantic descriptions. MotionScape is designed precisely to fill this gap, offering a more challenging and targeted foundation for training and evaluating world models in high-dynamic UAV environments.
\begin{table}[t]
\caption{Mean optical flow magnitude across different datasets.
A larger value indicates stronger motion variation and higher dynamicity.}
\centering
\small
\begin{tabular}{lc}
\toprule
\textbf{Dataset} & \textbf{Mean optical flow magnitude} \\
\midrule
VisDrone & 12.663 \\
UAVid & 9.745 \\
Tartanair & 41.856 \\
Zurich Urban MAV & 4.416 \\
\midrule
\textbf{Ours} & 36.630 \\
\bottomrule
\end{tabular}
\label{tab:flow_magnitude}
\end{table}
\begin{table}[t]
\caption{Quantitative results of different conditioning strategies for video continuation on Cosmos 2.5-2B.}
\centering
\small
\resizebox{\linewidth}{!}{
\begin{tabular}{lccccc}
\toprule
\textbf{Conditioning Strategy} & \textbf{PSNR$\uparrow$} & \textbf{SSIM$\uparrow$} & \textbf{LPIPS$\downarrow$} & \textbf{Warping error$\downarrow$} & \textbf{FVD$\downarrow$}\\
\midrule
No conditioning & 13.433 &0.739 &0.482 &0.019 &96.256\\
with caption & 13.292 &0.724 &0.492 &0.022 &96.905 \\
\midrule
with pose & 13.487 &0.742 &0.486 &0.021 &96.762 \\
\bottomrule
\end{tabular}}
\label{tab:baseline}
\end{table}
\section{Conclusion}
In this paper, we present MotionScape, a large-scale 4K UAV video dataset for high-dynamic world modeling. MotionScape not only covers real-world videos with complex highly dynamic 6-DoF motion, but also provides semantic descriptions and camera pose information aligned with video segments, thereby establishing a semantically and geometrically aligned foundation for training and evaluation. Experimental results show that MotionScape exhibits substantially stronger dynamicity than several existing real-world UAV datasets, while approaching the motion intensity of highly challenging synthetic data. Further generation experiments indicate that even for advanced video generation models such as Cosmos 2.5-2B, the ability to exploit auxiliary conditions, including textual descriptions and pose information, remains limited in high-dynamic UAV video continuation. This suggests that both current models and existing training data still fall short in supporting robust world modeling under complex 6-DoF motion. In future work, we plan to build upon MotionScape to explore training and control strategies for world models that are better suited to highly dynamic UAV scenarios.
%%
%% The acknowledgments section is defined using the "acks" environment
%% (and NOT an unnumbered section). This ensures the proper
%% identification of the section in the article metadata, and the
%% consistent spelling of the heading.
% \begin{acks}
% To Robert, for the bagels and explaining CMYK and color spaces.
% \end{acks}
%%
%% The next two lines define the bibliography style to be used, and
%% the bibliography file.
\bibliographystyle{ACM-Reference-Format}
\bibliography{acmart}
%%
%% If your work has an appendix, this is the place to put it.
% \appendix
\end{document}
\endinput
%%
%% End of file `sample-sigconf.tex'.