BuckTales : A multi-UAV dataset for multi-object tracking and re-identification of wild antelopes
Extracted source text · arXiv 2411.06896
\documentclass{article}
% if you need to pass options to natbib, use, e.g.:
% \PassOptionsToPackage{numbers, compress}{natbib}
% before loading neurips_2024
% ready for submission
%\usepackage{neurips_2024}
% to compile a preprint version, e.g., for submission to arXiv, add add the
% [preprint] option:
\usepackage[preprint]{neurips_2024}
% to compile a camera-ready version, add the [final] option, e.g.:
% \usepackage[final]{neurips_2024}
% to avoid loading the natbib package, add option nonatbib:
% \usepackage[nonatbib]{neurips_2024}
\usepackage[utf8]{inputenc} % allow utf-8 input
\usepackage[T1]{fontenc} % use 8-bit T1 fonts
\usepackage{hyperref} % hyperlinks
\usepackage{url} % simple URL typesetting
\usepackage{booktabs} % professional-quality tables (top-/mid-/bottomrule)
\usepackage{amsfonts} % blackboard math symbols
\usepackage{nicefrac} % compact symbols for 1/2, etc.
\usepackage{microtype} % microtypography
\usepackage{xcolor} % colors
\usepackage{graphicx}
\usepackage{multirow} % merge cell effect in the table
\usepackage{soul}
\title{BuckTales: A multi-UAV dataset for multi-object tracking and re-identification of wild antelopes}
% The \author macro works with any number of authors. There are two commands
% used to separate the names and addresses of multiple authors: \And and \AND.
%
% Using \And between authors leaves it to LaTeX to determine where to break the
% lines. Using \AND forces a line break at that point. So, if LaTeX puts 3 of 4
% authors names on the first line, and the last on the second line, try using
% \AND instead of \And before the third author name.
\author{%
Hemal Naik$^{1,2,3,4}$
\thanks{Corresponding authors: hmnaiks@gmail.com, vivekhsridhar@gmail.com}\\
%\thanks{Use footnote for providing further information
% about author (webpage, alternative address)---\emph{not} for acknowledging
% funding agencies.} \\
% \texttt{hnaik@ab.mpg.de} \\
% examples of more authors
\And
Junran Yang$^{1}$ \\
% \texttt{email} \\
\And
Dipin Das$^{1}$ \\
% \texttt{email} \\
\AND
Margaret C Crofoot$^{1,2,3}$ \\
% \texttt{email} \\
\And
Akanksha Rathore$^{1,2,4,5}$ \\
% Affiliation \\
% Address \\
% \texttt{email} \\
\And
Vivek Hari Sridhar$^{1,2,3,4}$ \\ \\
Department for the Ecology of Animal Societies, Max Planck Institute of Animal Behavior$^{1}$ \\
Center for the Advanced Study of Collective Behaviour, University of Konstanz$^{2}$ \\
Department of Biology, University of Konstanz$^{3}$ \\
Centre for Ecological Sciences, Indian Institute of Science$^{4}$ \\
Computer Science \& Information Systems, BITS Pilani, Hyderabad$^{5}$ \\
}
\begin{document}
\maketitle
\begin{abstract}
Understanding animal behaviour is central to predicting, understanding, and mitigating impacts of natural and anthropogenic changes on animal populations and ecosystems. However, the challenges of acquiring and processing long-term, ecologically relevant data in wild settings have constrained the scope of behavioural research. The increasing availability of Unmanned Aerial Vehicles (UAVs), coupled with advances in machine learning, has opened new opportunities for wildlife monitoring using aerial tracking.
However, limited availability of datasets with wild animals in natural habitats has hindered progress in automated computer vision solutions for long-term animal tracking.
Here we introduce \textit{BuckTales}, the first large-scale UAV dataset designed to solve multi-object tracking (MOT) and re-identification (Re-ID) problem in wild animals, specifically the mating behaviour (or lekking) of blackbuck antelopes. Collected in collaboration with biologists, the MOT dataset includes over 1.2 million annotations including 680 tracks across 12 high-resolution (5.4K) videos, each averaging 66 seconds and featuring 30 to 130 individuals. The Re-ID dataset includes 730 individuals captured with two UAVs simultaneously. The dataset is designed to drive scalable, long-term animal behaviour tracking using multiple camera sensors. By providing baseline performance with two detectors, and benchmarking several state-of-the-art tracking methods, our dataset reflects the real-world challenges of tracking wild animals in socially and ecologically relevant contexts. In making these data widely available, we hope to catalyze progress in MOT and Re-ID for wild animals, fostering insights into animal behaviour, conservation efforts, and ecosystem dynamics through automated, long-term monitoring.
\end{abstract}
\section{Introduction}
In a rapidly changing world, behaviour is the most flexible tool in an animal's toolkit to solve problems and adapt to novel challenges. While numerous observational and automated sensor-based techniques exist to study animal behaviour, the past decade has seen a significant increase in the use of animal tracking from camera-based technology \citep{couzin2023emerging,tuia2022perspectives}. The ability to record detailed movement and behavioural data of individual animals non-invasively, and over extended periods, has facilitated a range of discoveries across various aspects of animal lives—from the study of animal architecture \citep{smith2021imperfect} and the understanding of individual and collective animal decision-making \citep{sridhar2021geometry,sampaio2024multidimensional}, to revealing the mechanisms of social interactions that result in coordinated collective movement \citep{torney2018inferring,sridhar2023inferring} and anti-predatory strategies \citep{rosenthal2015revealing,sosna2019individual,davidson2021collective}.
While insights gained from vision-based animal tracking are on the rise, a lot of this work is still restricted to controlled laboratory settings \citep{naik20233d}. To translate these methods for research on animals in their natural habitats, biologists are beginning to leverage advances in aerial imagery (through the use of Unmanned Aerial Vehicles or UAVs) and state-of-the-art techniques in computer vision and machine learning to record and track freely-moving animals in the wild \citep{koger2023quantifying,price2023framework,ozogany2023fine,maeda2021aerial}.
However, the problem remains that most publicly available datasets of animal monitoring from UAVs do not offer ground truth for movement tracking problems.
Furthermore, animals in the wild do not restrict their movement to specific areas captured within field-of-view of a single UAV. Therefore, recording behaviours occurring in relatively large spatial scales requires robust tracking solutions and the fusion of information from multiple simultaneously-flying UAVs.
Here we present BuckTales, the first dataset designed to study the behaviour of wild animals (in this case, blackbuck) in a large area using multiple simultaneously-flying UAVs.
The dataset focuses on the task of UAV-based multi-object tracking (MOT) and re-identification (Re-ID) of wild animals.
The MOT dataset contains 22.5K frames (12.5 min) in total with the longest video >3 min (5805 frames) and average of 75 individuals per video.
The Re-ID dataset focuses on the task of merging the tracking data from multiple UAVs and thus annotations are provided for each antelope moving in the overlapping region of a UAV pair.
These data are collected in collaboration with behavioural biologists studying lekking--an extremely rare mating system observed in <2\% of mammalian species.
Thus, our dataset presents a unique opportunity where results from algorithms trained here may have cross-disciplinary and direct real-world applications.
This article is written for both biologists and computer scientists. Therefore, we provide details for creating such datasets and the relevant code base for biologists to replicate our process. Similarly, for the computer science community, we offer baseline experiments with commonly known detection and tracking methods to demonstrate limitations of state-of-the-art methods. An added contribution of our dataset is that it is accompanied by specific analyses that highlight its suitability to becoming a benchmark dataset for solving large scale animal tracking with multiple camera sensors.
\begin{figure}
\centering
\includegraphics[width=0.8\textwidth]{Schematic.jpg}
\caption{A schematic of the data collection strategy and dataset details. The image in the top displays top-down view of a blackbuck lek from a single drone with male territories marked. The close-ups on the right show an example male and female. The bottom figure is a simplified data collection scheme, shown here with two drones (note that the actual collection scheme involved three drones). Three types of annotations are made available with the manuscript: object detection, multi-object tracking and re-identification.}
\label{fig:schematic}
\end{figure}
\section{State of the Art}
It is hard to capture videos of detailed animal behaviour in the wild.
Readying a dataset that captures diverse animal behaviours requires extensive planning, logistical and financial support, and sizable human effort to record and annotate massive amounts of images and videos. With that in mind, computer scientists often turn to annotating images and videos widely available on the internet \citep{chen2023mammalnet}). While models trained on these datasets have greatly furthered our ability to detect animals under diverse backgrounds and environments, the methods themselves have limited portability to solve problems encountered by biologists. On the other hand, biologists capture large scale datasets of wild animals in ecologically relevant settings but invest little to create annotated datasets to facilitate method development by computer scientists. The lack of collaboration between the two fields has led to a dearth of high-quality datasets, resulting in few researchers working on challenging computer vision problems relevant to wildlife. In this section, we showcase the scarcity of datasets by highlighting the state-of-the-art datasets involving wild animals captured with UAVs in context of multi-object tracking (MOT) and re-identification (Re-ID). Additionally, we also point out relevant contributions of this paper.
\subsection{Multi-object tracking with UAVs}
Creating a good multi-object tracking dataset with wild animals is challenging. There are no standards for knowing what comprises a good MOT dataset. Various aspects like i) having long annotated sequences with multiple moving targets, ii) capturing variation in animal movement and behavioural characteristics, and iii) encompassing varied proximity to neighbours during different types of social interactions, all appear to be important. With that in mind, scraping videos off the internet is far from optimal as it does not guarantee that the dataset will account for the above-mentioned diversities essential for a good MOT dataset.
A predominant approach to the MOT problem is tracking by detection. Perhaps, this is why most UAV datasets of wild animals largely focus on the object detection problem (see Table \ref{tab:literature}). Even within the biological literature, researchers have proposed tracking by detection to track wild animals with UAVs \citep{koger2023quantifying,rathore2023multi}. The problem however persists that these (and other) approaches can not be evaluated for their tracking accuracy due to the lack of a benchmark dataset for multi-object tracking of animals in the wild. \citep{price2023framework} recently offered a fast method for annotating datasets for the task of automated behaviour recognition. The method makes use of multi-object tracking to find individuals in videos but the authors do not provide datasets with annotations for the MOT problem. \citep{bonetto2023synthetic} presented a novel idea using synthetic data to improve animal detection, thus reducing the need for large real-world datasets. However, such synthetic models for animal movement cannot reflect real movements of animals and thus have limited contribution towards the MOT problem.
One of the only animal MOT datasets with UAVs consists of cows grazing on a farmland \citep{barrios4442721monitoring}. The dataset offers a total of 106 seconds of annotated data split into 7 video sequences. Here, the main intention was to develop a method for tracking cattle with thermal cameras and the dataset contains relatively short segments which are not ideal for evaluating tracking performance over longer periods of time. Other MOT datasets include, terrestrial cameras in outdoor setting such as AnimalTrack \citep{zhang2023animaltrack}, semi-controlled \citep{waldmann20243d} or controlled settings \citep{naik20233d}. Generic MOT datasets such as TAO dataset (\citep{bai2021gmot}) and GMOT-40 \citep{dave2020tao} do contain some sequences with animals but arguably, the challenges of solving MOT from an aerial view are different and require specialised datasets that address this niche.
Finally, it should be noted that MOT datasets with UAVs designed for tracking humans or man-made objects in urban areas are also scarce \citep{zhu2021detection}. We provide some of the longest annotated sequences (>3 min) at the highest resolution (5.4K) and show (with further analysis) that longer trajectories allow greater opportunities to evaluate the robustness of tracking as multiple individuals interact with each other. Our dataset contains tracking challenges which directly contribute towards improving existing MOT solutions irrespective of the target object.
\subsection{Re-identification with UAVs}
The problem of identity tracking is highly relevant in the context of long-term monitoring of animals.
Re-identification is largely pursued with the intention of monitoring the same individuals between different days or even years. Image-based individual identification between multiple days is successfully demonstrated with some animals \citep{wahltinez2024open} in the wild but it is largely practiced in controlled environment such as lab studies \citep{romero2019idtracker, walter2021trex}. Re-identification algorithms use unique body patterns (whales \citep{berger2017wildbook},tigers \citep{li2019atrw}, cow \citep{li2022individual}) or face recognition like algorithms to assign identities e.g. chimps \citep{schofield2019chimpanzee, paulet2024deep}.
More recently, UAVs are also being deployed for individual identification in animal behaviour research \citep{ozogany2023fine, maeda2021aerial}. To the best of our knowledge, only two datasets are available for Re-ID of animals using UAV images. The first is a datasets with 88,000 images of 139 crocodiles \citep{desai2022identification} and the second is 706 images of 10 cows \citep{mucher2022detection}. Animals like blackbucks do not exhibit visibly unique patterns and thus creating a ground truth with human annotators is extremely challenging, even with terrestrial cameras (e.g. camera traps).
In this paper, we define the problem of re-identification as a problem of identifying the same individual recorded at the same time (Fig \ref{Fig:reid}) with two separate UAVs.
This approach does not require human annotators to identify individuals specifically and ground truth can be created by establishing identities between simultaneously recorded videos (see Sec. \ref{ss:reid}).
A similar approach was used for Re-ID of cars by \citep{wang2019vehicle} for the purpose aerial surveillance using multiple UAVs but such datasets do not exist for monitoring wild animals.
Our dataset is the first to tackle the Re-ID problem for a large group of animals using multiple UAVs and the first to provide video annotations for the Re-ID problem. We have defined the Re-ID problem in context of combining tracking data obtained from multiple sensors and this approach paves the way for solving large scale monitoring with a fleet of automated UAVs in the wild.
\section{BuckTales dataset}
This section contains details about the conceptualization of the dataset, the data collection protocol and the structure of the dataset. First, we inform the reader about lekking behaviour of blackbuck. Then we offer information about our data collection scheme which establishes relevance to the multi-object tracking and the Re-ID problem. The final section provides structure of the dataset along with the annotation scheme used for preparing the dataset.
\subsection{Behaviour}
Lekking is a relatively rare mating system in which males aggregate in closely clustered territories that females visit to find mates \citep{rathore2023lekking}.
Both the aggregation, and the site of its occurrence, are known as leks.
Fig \ref{fig:schematic} shows an aerial image and a schematic of a lek.
%serve as dynamic arenas of social interactions ranging from mating to competitive behaviours.
The circular spots seen in the lek are territorial markings of males.
Males often engage in agonistic interactions, including ritualistic displays, fights and chases, with their neighbours.
When females visit a territory, their interactions with the territory-owner can vary from courtship displays to mounting in a few seconds to several hours.
Females also move across territories assessing different males while being chased, displayed to, and courted.
With a wide array of social interactions in an ecologically and evolutionarily relevant context, leks serve as an excellent case study to address with multi-object tracking (MOT).
\subsection{Data collection scheme}
Videos of the lek were obtained at resolution of 5.4K and a frame rate of 30 fps with all UAVs flying at an altitude of 80 metres. This recording altitude was chosen based on previous UAV-based studies on blackbuck \citep{rathore2023multi} as it provides a balance between a broad field of view and the number of pixels per animal. Given the size of the lek ($\sim$250 m x 300 m), we employed three DJI Air 2S drones simultaneously to cover a majority of the lekking arena. All recordings were obtained with the cameras pointing at -90\textdegree (directly downward) to minimise occlusions. Fig \ref{fig:schematic} shows a schematic representation of our data collection strategy.
In total, we collected \(\sim\)10 TB of data, consisting of video sequences lasting between 30 and 120 minutes. These extended-duration recordings were achieved using a relay technique \citep{koger2023quantifying}, detailed in the supplementary material. The dataset does not include images of blackbuck outside the lekking area, nor does it capture blackbuck herds, as the species typically moves in groups within the same habitat.
Additionally, seasonal variations in coat color—such as the lighter coloration males exhibit outside the breeding season—are not reflected in the dataset.
Due to these factors, we do not anticipate the misuse of our dataset in contexts like hunting or poaching.
%. Additional details about recordings are provided in supplementary information.
\subsection{Dataset structure and annotation process}
We divided the dataset in three parts to focus on three different problems separately (See Fig \ref{fig:schematic}).
The first part consists of annotated images for object detection, the second part contains annotated videos for multi-object tracking, and the third part includes 11 pairs of videos from two simultaneously flying drones (with an overlap) for addressing the Re-ID problem.
\subsubsection{Object detection dataset}
\label{subss:OBJDet}
Object detection is a necessary prerequisite for designing successful tracking algorithms, especially for the tracking by detection approach.
%The videos for MOT annotations are selected with intention of involving high activity.
However, videos with annotations for the MOT do not reflect sufficient variation in images across different days of lek and thus are not representative of the entire behavioural dataset. Therefore we prepared a separate dataset to build a reliable blackbuck detection algorithm.
The intention is to use the same algorithm on the entire dataset collected for behavioural studies and the detection models trained on this dataset are used for evaluating different trackers on the MOT dataset (see Sec. \ref{ss:multiObj}).
The detection dataset consists of 320 images selected from nine different days and a total of 18.4K annotations of male and female blackbuck (details supplementary).
We use Labelbox to annotate the images with bounding boxes. The images are selected with careful consideration of factors such as time of the day (sunrise/sunset), the presence of shadows of the animals, the lighting conditions (weather) and most importantly, the presence of males and females on the lek.
\subsubsection{Multi-object tracking (MOT) dataset}
\label{ss:multiObj}
The MOT dataset focuses on the tracking of two class categories: blackbuck males and blackbuck females. All individuals are marked with a class label and unique individual IDs using the DarkLabel tool \footnote{https://github.com/darkpgmr/DarkLabel}. Each individual is manually selected in the first frame and DarkLabel automatically propagates the annotations to the next frame. The tool assigns a new bounding box to the individual while maintaining the identity and class information. A team of domain experts then verified the propagated annotations.
We also wrote additional scripts to identify manual errors in the annotations e.g. ID duplications or missing class assignments (see supplementary).
These scripts are provided with the dataset and will benefit others biologists who wish to replicate our approach.
We selected videos based on the variation in lekking activity.
The MOT dataset includes 12 video sequences annotated with a total number of 1.2 million bounding boxes and they consist an average 60 tracks per video.
Our longest video is 193 seconds (>5800 frames) and consists of 127 individuals.
We highlight that continuous annotations of such magnitude are not available with any existing MOT datasets including captive or wild animals.
Sec \ref{sec:AnalyseBench} contains more information on the quality of the dataset w.r.t. the MOT problem and supplementary material includes the annotation details of each video.
\begin{figure}
\centering
\includegraphics[width=0.8\textwidth]{reIdentification.jpg}
\caption{Sample images from the Re-ID dataset. The images in each column demonstrates the variation in appearance of an individual when captured from two different drones.}
\label{Fig:reid}
\end{figure}
\subsubsection{Re-identification (Re-ID) dataset}
\label{ss:reid}
Our data collection scheme involves multiple simultaneous drones with overlapping fields of view (see Fig \ref{fig:schematic}). Since individuals are tracked independently in each video, a re-identification strategy is required to fuse data in the overlap between two adjacent video streams.
This problem of data fusion is highly relevant for multi-sensor based studies, especially when data fusion is not possible with camera calibration or image stitching due to lack of significant overlap or unique matching features.
%Rev 2-3 changed
Preparing ground truth for re-identification problem is extremely challenging in the case of animals that appear identical.
Even field experts are unable to identify individuals blackbucks between different days.
We used a machine learning enabled annotation workflow to simplify re-identification of same individuals from two different video streams with slightly different appearance of the animal.
%to offer the first video dataset for Re-IDentification problem.
For annotation, we trained a YOLOv8 model using the detection dataset and used ByteTracker to assign tracking IDs to all individuals in each pair of videos.
Annotators then watched the two videos augmented with IDs (provided with dataset) and marked IDs of all individuals visiting the overlapping areas by using movement to establish their identity.
A summary file was then created with annotation of IDs from both videos mapped with corresponding frames numbers and their bounding box positions in raw videos.
Figure \ref{Fig:reid} shows example images of difference scenarios demonstrating challenges within this dataset.
Our dataset is the first of its kind, offering ground truth for re-identification of individuals in a scenario where domain experts are unable to do so.
The datasets consists of 730 tracks from 11 different video-pairs.
The average track length of each video is between 600-1100 frames, and the duration of the tracks account for the movement of the animal in the overlapping region only (See suppl. video).
It is possible to use the dataset for both one shot learning or temporal learning of individual features, especially in case of animals with similar appearance.
The problem given in this paper is technically an extension of feature based re-identification method used for tracking occluded targets within the same video.
\subsubsection{Data format \& availability}
The detection data is provided in the COCO and YOLOv8 format.
%We have provided additional scripts to convert the data in the YOLO format.
The MOT annotations are provided in the MOT challenge format \footnote{https://motchallenge.net/}.
%We release the tracking annotations in MOT challenge format for ease of use and evaluation.
The MOT data is labeled using DarkLabel tool with a customized format and this is also provided with the dataset.
The Re-ID dataset is provided in a customised format (details in supplementary). The dataset is publicly available as \href{https://doi.org/10.17617/3.JCZ9WK}{BuckTales} at Edmond platform. Further, links for relevant repositories are provided in the supplementary.
\section{Dataset analysis \& benchmarking}
\label{sec:AnalyseBench}
Blackbuck, being moderately sized and among the fastest terrestrial mammals (capable of running up to 80 km/h), present a broad range of tracking challenges relevant to the study of animal behaviour.
Hence, effectively tracking these antelopes could generalize to other similarly sized or larger mammals. In this section, we provide a cursory analysis of the dataset and discuss characteristics relevant to the MOT problem. We also provide baseline results of two state-of-the-art object detection methods and benchmarking for MOT with three different tracking approaches.
%We also present results of additional experiments done with the dataset to showcase best possible results with existing tracking and detection models.
\begin{figure}
\centering
\includegraphics[width=\textwidth]{DataAnalysis.jpg}
\caption{A cursory movement analysis highlighting the movement and behavioural diversity captured by our dataset. \textbf{A,B)} show diversity in individual specific behaviours while \textbf{C–G)} shows the same in pairs of interacting individuals.}
\label{Fig:Analysis}
\end{figure}
\subsection{Movement patterns in dataset}
\label{ss:movementpat}
We combined all annotations from different videos to showcase common trends and patterns related to the movement of male and female blackbuck in the annotated videos.
These analyses serve as indicators of the value this dataset adds with respect to MOT problem.
Each video contains individuals exhibiting different types of behaviours i.e. resting, walking, running, displaying, courting, fighting etc. and signatures of these behaviours are visible in their movement characteristics.
Below, we quantify distributions of various behaviours in which individuals engage and use pairwise distance and speed profiles to illustrate signatures of these behaviours in their movement.
\textbf{Length of annotations}: First, we computed the number of annotated frames per individual to see length profile of annotations. More than 90\% individuals in our dataset are annotated for over 300 frames (>10 seconds), and over 50\% individuals are annotated for more than 900 frames (>30 seconds). This variation in the tracking duration primarily occurs due to one of two reasons: i) Variation in the length of annotated video segments. The shortest video sequence contains 752 (\(\sim\)25 seconds) frames while the longest video sequence contains over 5800 frames (>3 min). ii) Duration for which the animal stays within the frame. Even within video sequences, some animals do not remain in the video for the entire length of the video and thus are annotated for a shorter duration (see Fig \ref{Fig:Analysis}A).
\textbf{Distance coverage}: In addition to the number of frames tracked, we measured each individual's distance covered in our dataset by measuring accumulated movement in pixels of bounding box center. (Fig \ref{Fig:Analysis}A; see supplementary information for the same plot split by video). While there are more males on the lek than females (68.7\% males and 31.3\% females), no obvious differences are noted in the distances they travel (Fig \ref{Fig:Analysis}A).
Our analyses show a wide range in the distance distribution of individuals and a strong correlation between the number of frames an individual is tracked and the distance it travels (Fig \ref{Fig:Analysis}A). We show that individuals that travel less can in part be accounted for by the short length of their tracks, highlighting the need for longer-term tracking to gain biological insights from these types of data.
\textbf{Speed}: Another individual-specific metric we compute is speed. In order to highlight the various behaviours that can be captured by this kinematic measure, we present the speed profile (Fig \ref{Fig:Analysis}B) of a sample individual that exhibits stationary, walking and running behaviours, all within a minute. These sudden transitions in individual speed profiles make movement-based tracking (for example using a Kalman filter) a challenge.
\textbf{Proximity}: After measuring individual movement metrics (distance covered and speed), we further quantified occasions where two individuals engaged in various types of interactions. As a first approximation of interactions, we quantified proximity between individuals by calculating two different metrics: i) The distribution of distances of each individual to its nearest neighbour. ii) IoU (Intersection over Union) of individual bounding boxes (Fig \ref{Fig:Analysis}C). Combining insights from these two measures, we show that most individuals have at least one other individual that is within 200 pixels and that our dataset includes \(\sim\)29k instances where at least two individuals have overlapping bounding boxes. These moments of close proximity capture various inter-individual interactions like displays, courtship and fighting (see supplementary). While these behaviours represent parts of the data that are of most interest to biologists, they also comprise moments with an increased probability of ID switching and loss of individual tracking.
\textbf{Inter-individual interactions}: Finally, we also visualise movement characteristics i.e., distance between or speed profiles of, pairs of individuals when they engage in the above-mentioned close proximity behaviours (Fig \ref{Fig:Analysis}D–G). Fig 3D shows the distance between two males displaying to each other (also referred to as a parallel walk). Figs \ref{Fig:Analysis}E–G show speed profiles of i) two males fighting (Fig \ref{Fig:Analysis}E), ii) a male chasing a female (Fig \ref{Fig:Analysis}F), and iii) a male courting a female (Fig \ref{Fig:Analysis}G). Once again, these behaviours represent biologically interesting moments within these data but also represent moments when the tracking is most likely to fail.
\subsection{Dataset Benchmarking}
This subsection reports performance of the state-of-the-art detection and tracking algorithms on our dataset.
%We used tracking by detection methods only and have not evaluated the performance of other tracking paradigms.
We trained object detection models using the object detection dataset and used the best-performing detector with various tracking algorithms for benchmarking.
Using a separate detection and tracking dataset allowed us to train a strong detector and leverage the entire MOT dataset for evaluation of the tracking results.
We have not provided a baseline for the Re-ID problem.
\subsubsection{Object detection}
We trained detectors based on two state-of-the-art object detection models: Fast-RCNN (through Detectron2 \citep{wu2019detectron2}) and YOLOv8 \citep{Jocher_Ultralytics_YOLO_2023}.
According to availability, NVIDIA A100 (40GB) and NVIDIA V100 (32GB) are used to train all our detection models.
We trained models with different image resolutions and different class configurations.
All details of the experiments are provided in the supplementary material with information of the train, validation, and test sets.
We used mAP as metric for selecting the best detector.
Our results show that YOLOv8m with image size of 5472 obtains the best mAP score of 0.624.
The detection dataset contains annotations for categories other than blackbuck i.e. birds, drones, unknown (dog) etc.
Our experiments suggest that the model trained only with male and female class categories performs better than one with all classes included, probably due to the rare occurrence of the other categories in our dataset. Another notable observation is that YOLOv8m with a halved image resolution trains 10x faster with minimal impact on the mAP results (see supplementary), which can be a practical consideration for processing large scale dataset with negligible performance loss.
\subsubsection{Multi-object tracking}
We evaluated state-of-the-art MOT tracking methods such as ByteTrack \citep{zhang2021bytetrack}, OC-SORT \citep{cao2023observation}, and BoT-SORT \citep{aharon2022bot}.
Four different pre-trained Re-ID models, trained with human data, are used to make appearance based tracking with BoT-SORT (details \ref{app:botsort}).
We used the default configuration in the BoxMOT library \footnote{https://github.com/mikel-brostrom/yolo\_tracking} and evaluated our results using TrackEval\footnote{https://github.com/JonathonLuiten/TrackEval}.
All of these approaches belong to the tracking by detection paradigm and use the detections from our the best performing YOLOv8 model (see supplementary).
We present HOTA, MOTA, MOTP, and IDF1 metrics in Table.\ref{tab:mot_baseline}, which are widely employed in the field as the main metrics to evaluate tracker performance. Our results show BoT-SORT as the best-performing tracker with the highest HOTA metric and the lowest number of ID switches.
This shows that models with feature based (Re-ID) tracking are better than localization based tracking approaches.
It should however be noted that Re-ID models require a dedicated GPU (in our case, the NVIDIA Quadro RTX 6000) whereas the other approaches do not require them.
% Rev 2
Our analysis highlights a known limitation of MOTA \citep{luiten2021hota}, which overlooks association errors, potentially leading to artificially high scores by permitting many identity switches.
For biologists, HOTA is a more appropriate metric as it better reflects the importance of accurate identity tracking over time.
This discrepancy is less evident in existing datasets with shorter tracks, where the likelihood of identity switches is lower.
%Another interesting insight is the discrepancy between HOTA and MOTA values, typically not seen with other datasets.
%MOTA is heavily biased towards detection performance and error in association is not penalized which results into high values of MOTA irrespective of high number ID switches.
%Meanwhile, HOTA remained moderate compared to other MOT datasets, probably due to its balanced measurement of both detection and association.
% Rev 2
%Blackbuck, being moderately sized and among the fastest terrestrial mammals (capable of running up to 80 km/h), presents a broad range of tracking challenges relevant to animal behavior studies. Effective tracking of blackbuck could generalize to other similarly sized or larger mammals. We show with movement analysis that our dataset includes a variety of interactions at different speeds and densities, providing a solid ground truth for tracking animals over relatively long trajectories.
\begin{table}
\centering
\small
\caption{Multi-object Tracking Benchmarking}
\begin{tabular}{ll|ccccccccccc}
\toprule
% header row with column names
Method
& Re-ID
& HOTA\textsuperscript{$\uparrow$}
& DetA\textsuperscript{$\uparrow$}
& AssA\textsuperscript{$\uparrow$}
& MOTA\textsuperscript{$\uparrow$}
& MOTP\textsuperscript{$\uparrow$}
& IDF1\textsuperscript{$\uparrow$}
& IDsw\textsuperscript{$\downarrow$} \\
% data rows
\midrule
\multirow{1}{*}{ByteTrack}
& - & 0.4950 & 0.5124 & 0.4868 & 0.9347 & 0.5932 & 0.8335 & 777 \\
\midrule
\multirow{1}{*}{OC-SORT}
& - & 0.4203 & 0.4880 & 0.3726 & 0.8699 & 0.5928 & 0.6421 & 8736 \\
\midrule
\multirow{4}{*}{BoT-SORT}
& - & 0.5286 & 0.5165 & 0.5495 & 0.9429 & 0.5933 & 0.9214 & 171 \\
& osnet\_x0\_25 & 0.5365 & 0.5159 & 0.5665 & 0.9430 & 0.5930 & 0.9418 & 158 \\ % Fastest - Same domain
& osnet\_x1\_0 & 0.5378 & 0.5160 & 0.5690 & 0.9429 & 0.5931 & 0.9451 & 153 \\
% Pre-trained for person Re-ID - Best performing -- Same domain re-ID (Using : https://kaiyangzhou.github.io/deep-person-reid/MODEL_ZOO.html) - Same domain
& osnet\_ain\_x1\_0 & 0.5369 & 0.5160 & 0.5671 & 0.9430 & 0.5931 & 0.9435 & 150 \\ % Best performing in cross domain
& lmbn\_n & 0.5355 & 0.5159 & 0.5644 & 0.9427 & 0.5929 & 0.9400 & 169 \\ % Box-MOT
\bottomrule
\label{tab:mot_baseline}
\end{tabular}
\end{table}
\section{Limitations}
\label{s:limitations}
In this section, first we will highlight the limitations of our dataset and then discuss the shortcomings of the detection and tracking models.
%We also suggest possible directions for future research in the context of solving both the MOT and Re-ID problems.
There are three main limitations to our dataset. First, we performed quality checks and improved several manual MOT annotations (see supplementary).
Despite these efforts, some bounding box annotations in our dataset do not reflect the exact size of the animals.
Tighter or looser bounding boxes were introduced during the pre-labelling stage, or during the MOT annotations with DarkLabel (See Fig \ref{fig:limitation} for examples). These variations themselves reduce the localisation IoU, in turn affecting the HOTA score. Second, sub-adult males and females are similar in appearance. Hence, the data may include a few misclassified individuals causing errors in detection and classification by the models (Fig \ref{fig:limitation}). However, most trackers do not consider the classification errors (male/female) during tracking assignment and seemingly assign the right identity ID based on the localization of bounding boxes (see Fig \ref{fig:limitation}).
Third, in the Re-ID dataset, we do not know the exact number of unique individuals in the dataset.
It is possible but unlikely that same individuals were present at same location between different days.
We hope to have captured sufficient variation in individuals by annotating videos across different days, in two different regions of the lek.
It is for this reason that we limit our Re-ID dataset to identifying individuals within video segments rather than between segments.
Next, we highlight limitations of the state-of-the-art methods in the context of our dataset. Several types of detection and tracking errors occur when individuals are in close proximity. This results in multiple individuals being detected as a single individual, loss of tracking, and identity switches (Fig \ref{fig:limitation}). This problem is reduced in feature based tracking methods, however, such considerations can improve localization based trackers. As mentioned above, localisation-based trackers do not consider class assignment while associating tracking results. This observation also reveals a limitation of the existing evaluation metrics such as HOTA and MOTA. These metrics do not handle multi-classes instances while evaluating multi-object tracking and therefore our evaluation will not reflect errors due to misclassifications. Evaluation for MOT in this paper is therefore done using a single category i.e., blackbuck.
% Rev 1 - 2 - 3
\section{Future directions}
Tracking and identifying wild animals using UAVs is a promising and rapidly evolving research direction. While our dataset addresses the MOT problem with a large group of animals, it is focused on a single species in a specific behavioural context. To build a more generalisable animal tracking approach, larger datasets that encompass a broader diversity of animals and habitats are essential. Solving the Re-ID problem for animals that look very similar poses a significant challenge. New ground truthing approaches are needed to consistently identify the same individuals across days, seasons, and even years.
Another key challenge is designing computationally efficient methods to process large volumes of high-resolution video while maintaining high accuracy. Our fastest model runs at 32 fps but has significant errors, while the best-performing model operates at 3 fps (see supplementary), which is insufficient for large-scale, long-term UAV-based monitoring programs. One potential solution to resolve this is involves training smaller models by splitting one high-resolution image into multiple images of lower resolution, an approach we will explore in the future.
Beyond new datasets and methods, improved evaluation metrics are necessary to assess the effectiveness of algorithms and dataset complexity in the context of behaviour monitoring. For example, while HOTA is useful for evaluating tracking over long periods and accounting for identity switches, it does not accommodate multiple classes in MOT.
Consequently, we simplified the classification of males and females into a single blackbuck class for evaluation. Additionally, more advanced analyses techniques are needed to assess the complexity of datasets within the framework of computer vision challenges.
As shown in Sec.\ref{sec:AnalyseBench}, we use movement analysis to demonstrate that our dataset captures various types of animal movements.
Standardized analysis techniques will help identify gaps in datasets and provide a systematic method for capturing a wider range of behavioural patterns.
\section{Conclusion}
BuckTales is first large-scale UAV dataset to address multi-object tracking (MOT) and animal re-Identification (Re-ID) problem for studying wild animals in natural environments.
These data were collected with biologists using a data collection scheme using multiple simultaneously-flying UAVs. % and present data of a rare mating system.
We claim that our dataset captures the whole range of behaviours exhibited by blackbuck as part of a rare mating strategy and thus it is suitable to address the MOT problem with wild animals.
%will directly benefit others studying the species, similar mating behaviours, or those trying to address the MOT problem in wild animals.
We back these claims with specific movement analyses and benchmarking of existing trackers.
BuckTales contributes some of the longest sequences of video annotations with large number of wild animals for MOT and Re-ID problem.
%The dataset presents realistic scenario and it is prepared in close collaboration with biologists.
While our dataset contains a unique behaviour, we highlight that similar situations occur throughout the animal kingdom where large groups of animals occupy a large area for limited or extended periods.
With the rapid adoption of UAVs within the animal behaviour community, we expect that methodologies developed with our dataset will directly impact wildlife monitoring, conservation and research, and pave to way towards the adoption of a fleet of drones for long-term animal tracking in large open ecosystems.
\section{Acknowledgement}
Support for this project is provided by the Max planck society, Deutsche Forschungsgemeinschaft (DFG) under Germany’s Excellence Strategy EXC 2117-422037984 and by the Alexander von Humboldt Professorship endowed by the Federal Ministry of Education and Research awarded to Margaret C Crofoot. We thank Sakshi Rao, Mansi Dave, Kuldeep Chouhan, Aporoopa S R, Shreya William, Ritika Chaterjee, Binay Aswal for annotations and Elisabeth Böker for the quirky title idea.
% \begin{table}
% \caption{Sample table title}
% \label{sample-table}
% \centering
% \begin{tabular}{lll}
% \toprule
% \multicolumn{2}{c}{Part} \\
% \cmidrule(r){1-2}
% Name & Description & Size ($\mu$m) \\
% \midrule
% Dendrite & Input terminal & $\sim$100 \\
% Axon & Output terminal & $\sim$10 \\
% Soma & Cell body & up to $10^6$ \\
% \bottomrule
% \end{tabular}
% \end{table}
\newpage
{\small
\bibliographystyle{abbrvnat}
%\setcitepstyle{authoryear,open={((},close={))}}
\bibliography{biblio}
}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\newpage
\appendix
\section{Appendix / supplemental material}
\subsection{Dataset availability}
The dataset can be accessed using the link: \href{https://keeper.mpdl.mpg.de/d/a0df96c0088942f7b89d/}{Dataset}.
The existing structure is shared in three different folders. Each of them contain a README file which explains the structure.
We have provided datasets in standard format (detection:COCO, MOT: MOT challenge format) which makes it easier for other to load dataset with standard libraries.
Our plan is to submit the entire dataset on publicly available platform such as Zenodo.
The code-base used for various parts of the dataset will be provided separately through github links, also uploaded via Zenodo having a permanent code base.
\subsection{Limitations of dataset}
The figure provided in this section is to support the claim about limitations in the dataset and further explanation of some of the errors in tracking results.
We identified some of these error while doing qualitative observation of the results from best performing models.
\begin{figure}[h]
\centering
\includegraphics[width=\textwidth]{limitation_challenges.jpg}
\caption{The figure highlighs examples of some limitations of the existing dataset and the state of the art. Images on the left- who that bounding box detection does not always fit perfectly. Shadows and difficult postures can produce missing detections or create confusion with identification of class category. Proximity of the animals also contributes to error in tracking due to missing detection, merged bounding boxes or switching in IDs. }
\label{fig:limitation}
\end{figure}
\subsection{Dataset comparison}
The table provided in this section covers all the dataset available with UAV based images. There are other datasets with annotations of points for counting animals, however they are not directly compatible with bounding boxes and we have not considered them in this comparison table.
Similarly, there are few datasets from other aerial imaging modalities like planes or satellite.
We have not included these as well to keep the comparisons concise with drone based imaging.
% \begin{table}[h]
% \caption{Details of datasets with animals and UAVs}
% \label{tab:literature}
% \centering
% \begin{tabular}{llllllll}%{width=\textwidth}
% \toprule
% % \multicolumn{2}{c}{Part} \\
% % \cmidrule(r){1-2}
% \multicolumn{8}{c}{Object detection problem} \\ \midrule
% Dataset & Habitat & Categories & Annotations & Type & Images & Pixels & Height (m) \\
% \midrule
% % Hu & Wetland & Multiple & 8976 & Thermal & 541 & 7 & 100\\
% \citep{weinstein2022general} & Wetland & Multiple & 23765 & RGB & 23765 & 35 & 76-91 \\
% \citep{shao2020cattle} & Cattle pasture & Single & 1919 & RGB & 664 & 90 & 50 \\
% % NOAA & Coastline & Seals & 14311 & RGB + IR & 44185 & 55 & Unknown \\
% \citep{bondi2020birdsai} & Grassland & Multiple & 166221 & Thermal & 61994 & 35 & 60-120 \\
% \textbf{Ours} & Grassland & Multiple & 18488 & RGB & 320 & 1600 & 80 \\
% \midrule
% \multicolumn{8}{c}{MOT Problem} \\
% \midrule
% Dataset & Habitat & Categories & Tracks & Type & Images & Pixels & Height (m) \\
% \midrule
% \citep{barrios4442721monitoring} & Grass & Single & 241 & Thermal & 959 & X & Var. \\
% \textbf{Ours} & Grassland & Multiple & 680 & RGB & 22502 & 1600 & 80 \\
% \midrule
% \multicolumn{8}{c}{Re-ID problem} \\ \midrule
% Dataset & Habitat & Categories & Instances & Type & Images & Pixels & Height (m) \\
% \midrule
% \citep{desai2022identification} & River, Pond & Single & 143 & RGB & 88000 & 1000 & 8-10 \\
% \textbf{Ours} & Grassland & Multiple & 730 & RGB & 589828 & 1600 & 80 \\
% \bottomrule
% \end{tabular}
% \end{table}
\begin{table}[h]
\caption{Details of datasets with animals and UAVs}
\centering
\begin{tabular}{ccccccc}%{width=\textwidth}
\toprule
% \multicolumn{2}{c}{Part} \\
% \cmidrule(r){1-2}
\multicolumn{7}{c}{Object detection problem} \\ \midrule
Dataset & Categories & Annotations & Type & Images & Pixels & Height (m) \\
\midrule
% Hu & Wetland & Multiple & 8976 & Thermal & 541 & 7 & 100\\
\citep{weinstein2022general} & Multiple & 23765 & RGB & 23765 & 35 & 76-91 \\
\citep{shao2020cattle} & Single & 1919 & RGB & 664 & 90 & 50 \\
% NOAA & Coastline & Seals & 14311 & RGB + IR & 44185 & 55 & Unknown \\
\citep{bondi2020birdsai} & Multiple & 166221 & Thermal & 61994 & 35 & 60-120 \\
\textbf{BuckTales} & Multiple & 18488 & RGB & 320 & 1600 & 80 \\
\midrule
\multicolumn{7}{c}{MOT problem} \\
\midrule
Dataset & Categories & Tracks & Type & Images & Pixels & Height (m) \\
\midrule
\citep{barrios4442721monitoring} & Single & 241 & Thermal & 959 & X & Var. \\
\textbf{BuckTales} & Multiple & 680 & RGB & 22502 & 1600 & 80 \\
\midrule
\multicolumn{7}{c}{Re-ID problem} \\ \midrule
Dataset & Categories & Instances & Type & Images & Pixels & Height (m) \\
\midrule
\citep{desai2022identification} & Single & 143 & RGB & 88000 & 1000 & 8-10 \\
\textbf{BuckTales} & Multiple & 730 & RGB & 589828 & 1600 & 80 \\
\bottomrule
\end{tabular}
\label{tab:literature}
\end{table}
\subsection{BoT-SORT with Re-ID networks}
\label{app:botsort}
We used BoT-SORT as one of the methods which uses not only position but features of the detected targets to perform tracking.
There models arguably perform better against the only localization based trackers.
All the models used with Re-ID support were trained with human data for Re-ID task and we have not trained them specifically on our dataset.
The models are downloaded from torchreid \footnote{https://kaiyangzhou.github.io/deep-person-reid/MODEL\_ZOO.html} and the selection of the model is based on the performance of the models based on same-domain or cross-domain task.
Model \textit{osnet\_x0\_25} is the best performing model in same-domain Re-ID where as \textit{osnet\_x1\_0} model is the fastest Re-ID models with same-domain training.
\textit{osnet\_ain\_x1\_0} is one of the best perofmring model on the cross-domain Re-ID task.
% \subsection{Datasheet}
% \subsection{Dataset details}
% % Image selection
% \begin{table}[h!]
% \centering
% \caption{Sequence and Count}
% \label{tab:det_data_src}
% \begin{tabular}{lclc}
% \toprule
% Sequence & Count & Sequence & Count \\
% \midrule
% 20230303\_SE\_Lek1\_P2D3\_DJI\_0949 & 5 & 20230311\_SE\_Lek1\_P1D2\_DJI\_0959 & 15 \\
% 20230308\_SM\_Lek1\_P1D1\_DJI\_0060 & 5 & 20230311\_SE\_Lek1\_P2D4\_DJI\_0884 & 15 \\
% 20230303\_SE\_Lek1\_P3D5\_DJI\_0205 & 5 & 20230311\_SE\_Lek1\_P3D6\_DJI\_0869 & 15 \\
% 20230303\_SE\_Lek1\_P1D2\_DJI\_0691 & 5 & 20230312\_SM\_Lek1\_P1D1\_DJI\_0249 & 10 \\
% 20230308\_SM\_Lek1\_P2D3\_DJI\_0092 & 5 & 20230312\_SM\_Lek1\_P1D1\_DJI\_0266 & 10 \\
% 20230309\_SE\_Lek1\_P1D1\_DJI\_0127 & 10 & 20230312\_SM\_Lek1\_P2D3\_DJI\_0264 & 10 \\
% 20230308\_SM\_Lek1\_P3D6\_DJI\_0722 & 5 & 20230312\_SM\_Lek1\_P2D3\_DJI\_0287 & 10 \\
% 20230309\_SE\_Lek1\_P2D4\_DJI\_0797 & 10 & 20230312\_SM\_Lek1\_P3D5\_DJI\_0541 & 10 \\
% 20230309\_SE\_Lek1\_P3D5\_DJI\_0412 & 10 & 20230313\_SE\_Lek1\_P1D2\_DJI\_0027 & 15 \\
% 20230314\_SE\_Lek1\_P1D2\_DJI\_0061 & 10 & 20230313\_SE\_Lek1\_P2D4\_DJI\_0938 & 15 \\
% 20230314\_SE\_Lek1\_P2D4\_DJI\_0994 & 15 & 20230313\_SE\_Lek1\_P3D6\_DJI\_0937 & 10 \\
% 20230314\_SE\_Lek1\_P3D6\_DJI\_0973 & 10 & 20230314\_SM\_Lek1\_P1D1\_DJI\_0327 & 15 \\
% 20230315\_SM\_Lek1\_P1D1\_DJI\_0361 & 15 & 20230314\_SM\_Lek1\_P2D3\_DJI\_0352 & 15 \\
% 20230315\_SM\_Lek1\_P2D3\_DJI\_0388 & 10 & 20230314\_SM\_Lek1\_P3D5\_DJI\_0597 & 10 \\
% 20230315\_SM\_Lek1\_P3D5\_DJI\_0643 & 10 & 20230317\_SM\_Lek1\_P1D1\_DJI\_0393 & 5 \\
% 20230317\_SM\_Lek1\_P1D1\_DJI\_0393 & 5 & 20230317\_SM\_Lek1\_P2D3\_DJI\_0409 & 5 \\
% 20230317\_SM\_Lek1\_P2D3\_DJI\_0409 & 5 & 20230317\_SM\_Lek1\_P3D5\_DJI\_0672 & 5 \\
% \bottomrule
% \end{tabular}
% \end{table}
% % Table on images
% \begin{table}[h!]
% \caption{Image Distribution Across Different Classes}
% \label{tab:det_split}
% \centering
% \begin{tabular}{lccccc}
% \toprule
% Class & \makecell{Total\\ (320 images)} & \makecell{Train\\ (219 images)} & \makecell{Val\\ (50 images)} & \makecell{Test\\ (51 images)} & Actual Ratio \\
% \midrule
% bbfemale & 4481 & 3119 & 786 & 576 & 0.7 : 0.18 : 0.13 \\
% bbmale & 14002 & 9625 & 2249 & 2128 & 0.69 : 0.16 : 0.15 \\
% shadow & 2388 & 1585 & 379 & 424 & 0.66 : 0.16 : 0.18 \\
% drone & 50 & 34 & 8 & 8 & 0.68 : 0.16 : 0.16 \\
% bird & 70 & 47 & 10 & 13 & 0.67 : 0.14 : 0.19 \\
% unknown & 102 & 70 & 15 & 17 & 0.69 : 0.15 : 0.17 \\
% \bottomrule
% \end{tabular}
% \end{table}
% % Table of videos
% % Table on Re-ID annotations
% \subsection{MOT Annotations with DarkLabel}
% \label{Supp:MOT}
% \subsection{Detection experiments}
%\section{Invid}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\newpage
\section*{NeurIPS Paper Checklist}
% %%% BEGIN INSTRUCTIONS %%%
% The checklist is designed to encourage best practices for responsible machine learning research, addressing issues of reproducibility, transparency, research ethics, and societal impact. Do not remove the checklist: {\bf The papers not including the checklist will be desk rejected.} The checklist should follow the references and follow the (optional) supplemental material. The checklist does NOT count towards the page
% limit.
% Please read the checklist guidelines carefully for information on how to answer these questions. For each question in the checklist:
% \begin{itemize}
% \item You should answer \answerYes{}, \answerNo{}, or \answerNA{}.
% \item \answerNA{} means either that the question is Not Applicable for that particular paper or the relevant information is Not Available.
% \item Please provide a short (1–2 sentence) justification right after your answer (even for NA).
% % \item {\bf The papers not including the checklist will be desk rejected.}
% \end{itemize}
% {\bf The checklist answers are an integral part of your paper submission.} They are visible to the reviewers, area chairs, senior area chairs, and ethics reviewers. You will be asked to also include it (after eventual revisions) with the final version of your paper, and its final version will be published with the paper.
% The reviewers of your paper will be asked to use the checklist as one of the factors in their evaluation. While "\answerYes{}" is generally preferable to "\answerNo{}", it is perfectly acceptable to answer "\answerNo{}" provided a proper justification is given (e.g., "error bars are not reported because it would be too computationally expensive" or "we were unable to find the license for the dataset we used"). In general, answering "\answerNo{}" or "\answerNA{}" is not grounds for rejection. While the questions are phrased in a binary way, we acknowledge that the true answer is often more nuanced, so please just use your best judgment and write a justification to elaborate. All supporting evidence can appear either in the main paper or the supplemental material, provided in appendix. If you answer \answerYes{} to a question, in the justification please point to the section(s) where related material for the question can be found.
% IMPORTANT, please:
% \begin{itemize}
% \item {\bf Delete this instruction block, but keep the section heading ``NeurIPS paper checklist"},
% \item {\bf Keep the checklist subsection headings, questions/answers and guidelines below.}
% \item {\bf Do not modify the questions and only use the provided macros for your answers}.
% \end{itemize}
% %%% END INSTRUCTIONS %%%
\begin{enumerate}
\item {\bf Claims}
\item[] Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
\item[] Answer: \answerYes{} % Replace by \answerYes{}, \answerNo{}, or \answerNA{}.
\item[] Justification: We include claims about size of the dataset, provide clear context about motivation for data collection and the computer vision problem associated with the dataset.
% \item[] Guidelines:
% \begin{itemize}
% \item The answer NA means that the abstract and introduction do not include the claims made in the paper.
% \item The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
% \item The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalise to other settings.
% \item It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.
% \end{itemize}
\item {\bf Limitations}
\item[] Question: Does the paper discuss the limitations of the work performed by the authors?
\item[] Answer: \answerYes{} % Replace by \answerYes{}, \answerNo{}, or \answerNA{}.
\item[] Justification: Section 5 is dedicated to discussion about the limitations of this work. We have also pointed out limitations of methods and limitations of the evaluation metrics themselves.
% \item[] Guidelines:
% \begin{itemize}
% \item The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
% \item The authors are encouraged to create a separate "Limitations" section in their paper.
% \item The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
% \item The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
% \item The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
% \item The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
% \item If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
% \item While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognise that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalise honesty concerning limitations.
% \end{itemize}
\item {\bf Theory Assumptions and Proofs}
\item[] Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?
\item[] Answer: \answerYes{} % Replace by \answerYes{}, \answerNo{}, or \answerNA{}.
\item[] Justification: We do not report theoretical results in the dataset. We have provided assumptions regarding the analysis shown in the paper. %\justificationTODO{}
%\item[] Guidelines:
% \begin{itemize}
% \item The answer NA means that the paper does not include theoretical results.
% \item All the theorems, formulas, and proofs in the paper should be numbered and cross-referenced.
% \item All assumptions should be clearly stated or referenced in the statement of any theorems.
% \item The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.
% \item Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
% \item Theorems and Lemmas that the proof relies upon should be properly referenced.
% \end{itemize}
\item {\bf Experimental Result Reproducibility}
\item[] Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
\item[] Answer: \answerYes{} % Replace by \answerYes{}, \answerNo{}, or \answerNA{}.
\item[] Justification: We have provided sufficient details for reproduction of the dataset. We have give the checklist for datasets with further details. The analysis and benchmarking is done using existing state of the art. We have included all text related to our experiments and parameter settings in the supplementary along with main information in the article.
% \item[] Guidelines:
% \begin{itemize}
% \item The answer NA means that the paper does not include experiments.
% \item If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
% \item If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
% \item Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
% \item While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
% \begin{enumerate}
% \item If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.
% \item If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.
% \item If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
% \item We recognise that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
% \end{enumerate}
% \end{itemize}
\item {\bf Open access to data and code}
\item[] Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?
\item[] Answer: \answerYes{} % Replace by \answerYes{}, \answerNo{}, or \answerNA{}.
\item[] Justification: The data and related code are freely available. The dataset can be accessed as \href{https://doi.org/10.17617/3.JCZ9WK}{BuckTales} on Edmond service of Max Planck society and relevant repositories for analysis of annotations on github as \href{https://github.com/projectmela/Annotation_analysis_tools}{Annotation Analysis Tool}. The dataset is given with annotation tool \href{https://github.com/darkpgmr/DarkLabel}{DarkLabel} %\justificationTODO{}
% \item[] Guidelines:
% \begin{itemize}
% \item The answer NA means that paper does not include experiments requiring code.
% \item Please see the NeurIPS code and data submission guidelines (\url{https://nips.cc/public/guides/CodeSubmissionPolicy}) for more details.
% \item While we encourage the release of code and data, we understand that this might not be possible, so “No” is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).
% \item The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (\url{https://nips.cc/public/guides/CodeSubmissionPolicy}) for more details.
% \item The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
% \item The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.
% \item At submission time, to preserve anonymity, the authors should release anonymised versions (if applicable).
% \item Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.
% \end{itemize}
\item {\bf Experimental Setting/Details}
\item[] Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimiser, etc.) necessary to understand the results?
\item[] Answer: \answerYes{} % Replace by \answerYes{}, \answerNo{}, or \answerNA{}.
\item[] Justification: Major details regarding the use of algorithms is shared in the main text for understanding the result. MOT algorithms were not really trained with the dataset and thus details discussion on parameter tuning, data splits or hyper parameters is provided for the detection algorithms in the supplementary.
% \item[] Guidelines:
% \begin{itemize}
% \item The answer NA means that the paper does not include experiments.
% \item The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.
% \item The full details can be provided either with the code, in appendix, or as supplemental material.
% \end{itemize}
\item {\bf Experiment Statistical Significance}
\item[] Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
\item[] Answer: \answerYes{} % Replace by \answerYes{}, \answerNo{}, or \answerNA{}.
\item[] Justification: We have included graphs with complete distributions of the data.
% \item[] Guidelines:
% \begin{itemize}
% \item The answer NA means that the paper does not include experiments.
% \item The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
% \item The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
% \item The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
% \item The assumptions made should be given (e.g., Normally distributed errors).
% \item It should be clear whether the error bar is the standard deviation or the standard error of the mean.
% \item It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96\% CI, if the hypothesis of Normality of errors is not verified.
% \item For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).
% \item If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
% \end{itemize}
\item {\bf Experiments Compute Resources}
\item[] Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
\item[] Answer: \answerYes{} % Replace by \answerYes{}, \answerNo{}, or \answerNA{}.
\item[] Justification: We have provided details of the computing resources and also performance of models w.r.t time of inference. Other experiments are mentioned in the supplementary material.
% \item[] Guidelines:
% \begin{itemize}
% \item The answer NA means that the paper does not include experiments.
% \item The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
% \item The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
% \item The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).
% \end{itemize}
\item {\bf Code Of Ethics}
\item[] Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics \url{https://neurips.cc/public/EthicsGuidelines}?
\item[] Answer: \answerYes{} % Replace by \answerYes{}, \answerNo{}, or \answerNA{}.
\item[] Justification: The authors provide dataset with wild animals and thus ethical regulations are extremely important. We have provided all details related to this work and permission obtained from ethics committee for conducting the study.
% \item[] Guidelines:
% \begin{itemize}
% \item The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
% \item If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.
% \item The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).
% \end{itemize}
\item {\bf Broader Impacts}
\item[] Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
\item[] Answer: \answerYes{} % Replace by \answerYes{}, \answerNo{}, or \answerNA{}.
\item[] Justification: Our paper is motivated with the intent of offering large scale social impact. We have highlighted the role of our work in the context of its impact on research, conservation and even public policies.
% \item[] Guidelines:
% \begin{itemize}
% \item The answer NA means that there is no societal impact of the work performed.
% \item If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
% \item Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
% \item The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
% \item The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
% \item If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
% \end{itemize}
\item {\bf Safeguards}
\item[] Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
\item[] Answer: \answerYes{} % Replace by \answerYes{}, \answerNo{}, or \answerNA{}.
\item[] Justification: The dataset is carefully curated with all considerations to responsibilities towards protection of the animals. We declare to find no possible avenue of misuse for the given dataset.
% \item[] Guidelines:
% \begin{itemize}
% \item The answer NA means that the paper poses no such risks.
% \item Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
% \item Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
% \item We recognise that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.
% \end{itemize}
\item {\bf Licenses for existing assets}
\item[] Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
\item[] Answer: \answerYes{} % Replace by \answerYes{}, \answerNo{}, or \answerNA{}.
\item[] Justification: The assets are not publicly available at this stage due to ongoing biological study. They will be made available with CC-BY 4.0 License upon acceptance
% \item[] Guidelines:
% \begin{itemize}
% \item The answer NA means that the paper does not use existing assets.
% \item The authors should citep the original paper that produced the code package or dataset.
% \item The authors should state which version of the asset is used and, if possible, include a URL.
% \item The name of the license (e.g., CC-BY 4.0) should be included for each asset.
% \item For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
% \item If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, \url{paperswithcode.com/datasets} has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.
% \item For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
% \item If this information is not available online, the authors are encouraged to reach out to the asset's creators.
% \end{itemize}
\item {\bf New Assets}
\item[] Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
\item[] Answer: \answerYes{} % Replace by \answerYes{}, \answerNo{}, or \answerNA{}.
\item[] Justification: We have included documentation of all assests given in the dataset.
% \item[] Guidelines:
% \begin{itemize}
% \item The answer NA means that the paper does not release new assets.
% \item Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
% \item The paper should discuss whether and how consent was obtained from people whose asset is used.
% \item At submission time, remember to anonymise your assets (if applicable). You can either create an anonymised URL or include an anonymised zip file.
% \end{itemize}
\item {\bf Crowdsourcing and Research with Human Subjects}
\item[] Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
\item[] Answer: \answerNA{} % Replace by \answerYes{}, \answerNo{}, or \answerNA{}.
\item[] Justification: No crowd sourcing or working with human subjects.
% \item[] Guidelines:
% \begin{itemize}
% \item The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
% \item Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.
% \item According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
% \end{itemize}
\item {\bf Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects}
\item[] Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
\item[] Answer: \answerNA{} % Replace by \answerYes{}, \answerNo{}, or \answerNA{}.
\item[] Justification: We do not have human subjects or crowd sourcing.
% \item[] Guidelines:
% \begin{itemize}
% \item The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
% \item Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
% \item We recognise that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
% \item For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
% \end{itemize}
\end{enumerate}
\end{document}
\begin{thebibliography}{45}
\providecommand{\natexlab}[1]{#1}
\providecommand{\url}[1]{\texttt{#1}}
\expandafter\ifx\csname urlstyle\endcsname\relax
\providecommand{\doi}[1]{doi: #1}\else
\providecommand{\doi}{doi: \begingroup \urlstyle{rm}\Url}\fi
\bibitem[Aharon et~al.(2022)Aharon, Orfaig, and Bobrovsky]{aharon2022bot}
N.~Aharon, R.~Orfaig, and B.-Z. Bobrovsky.
\newblock Bot-sort: Robust associations multi-pedestrian tracking.
\newblock \emph{arXiv preprint arXiv:2206.14651}, 2022.
\bibitem[Bai et~al.(2021)Bai, Cheng, Chu, Liu, Zhang, and Ling]{bai2021gmot}
H.~Bai, W.~Cheng, P.~Chu, J.~Liu, K.~Zhang, and H.~Ling.
\newblock Gmot-40: A benchmark for generic multiple object tracking.
\newblock In \emph{Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition}, pages 6719--6728, 2021.
\bibitem[Barrios et~al.(2024)Barrios, Valente, and van Langevelde]{barrios4442721monitoring}
D.~B. Barrios, J.~Valente, and F.~van Langevelde.
\newblock Monitoring mammalian herbivores via convolutional neural networks implemented on thermal uav imagery.
\newblock \emph{Computers and Electronics in Agriculture}, 218:\penalty0 108713, 2024.
\bibitem[Berger-Wolf et~al.(2017)Berger-Wolf, Rubenstein, Stewart, Holmberg, Parham, Menon, Crall, Van~Oast, Kiciman, and Joppa]{berger2017wildbook}
T.~Y. Berger-Wolf, D.~I. Rubenstein, C.~V. Stewart, J.~A. Holmberg, J.~Parham, S.~Menon, J.~Crall, J.~Van~Oast, E.~Kiciman, and L.~Joppa.
\newblock Wildbook: Crowdsourcing, computer vision, and data science for conservation.
\newblock \emph{arXiv preprint arXiv:1710.08880}, 2017.
\bibitem[Bondi et~al.(2020)Bondi, Jain, Aggrawal, Anand, Hannaford, Kapoor, Piavis, Shah, Joppa, Dilkina, et~al.]{bondi2020birdsai}
E.~Bondi, R.~Jain, P.~Aggrawal, S.~Anand, R.~Hannaford, A.~Kapoor, J.~Piavis, S.~Shah, L.~Joppa, B.~Dilkina, et~al.
\newblock Birdsai: A dataset for detection and tracking in aerial thermal infrared videos.
\newblock In \emph{Proceedings of the IEEE/CVF Winter conference on applications of computer vision}, pages 1747--1756, 2020.
\bibitem[Bonetto and Ahmad(2023)]{bonetto2023synthetic}
E.~Bonetto and A.~Ahmad.
\newblock Synthetic data-based detection of zebras in drone imagery.
\newblock In \emph{2023 European Conference on Mobile Robots (ECMR)}, pages 1--8. IEEE, 2023.
\bibitem[Cao et~al.(2023)Cao, Pang, Weng, Khirodkar, and Kitani]{cao2023observation}
J.~Cao, J.~Pang, X.~Weng, R.~Khirodkar, and K.~Kitani.
\newblock Observation-centric sort: Rethinking sort for robust multi-object tracking.
\newblock In \emph{Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition}, pages 9686--9696, 2023.
\bibitem[Chen et~al.(2023)Chen, Hu, Coker, Berumen, Costelloe, Beery, Rohrbach, and Elhoseiny]{chen2023mammalnet}
J.~Chen, M.~Hu, D.~J. Coker, M.~L. Berumen, B.~Costelloe, S.~Beery, A.~Rohrbach, and M.~Elhoseiny.
\newblock Mammalnet: A large-scale video benchmark for mammal recognition and behavior understanding.
\newblock In \emph{Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition}, pages 13052--13061, 2023.
\bibitem[Couzin and Heins(2023)]{couzin2023emerging}
I.~D. Couzin and C.~Heins.
\newblock Emerging technologies for behavioral research in changing environments.
\newblock \emph{Trends in Ecology \& Evolution}, 38\penalty0 (4):\penalty0 346--354, 2023.
\bibitem[Dave et~al.(2020)Dave, Khurana, Tokmakov, Schmid, and Ramanan]{dave2020tao}
A.~Dave, T.~Khurana, P.~Tokmakov, C.~Schmid, and D.~Ramanan.
\newblock Tao: A large-scale benchmark for tracking any object.
\newblock In \emph{Computer Vision--ECCV 2020: 16th European Conference, Glasgow, UK, August 23--28, 2020, Proceedings, Part V 16}, pages 436--454. Springer, 2020.
\bibitem[Davidson et~al.(2021)Davidson, Sosna, Twomey, Sridhar, Leblanc, and Couzin]{davidson2021collective}
J.~D. Davidson, M.~M. Sosna, C.~R. Twomey, V.~H. Sridhar, S.~P. Leblanc, and I.~D. Couzin.
\newblock Collective detection based on visual information in animal groups.
\newblock \emph{Journal of the Royal Society Interface}, 18\penalty0 (180):\penalty0 20210142, 2021.
\bibitem[Desai et~al.(2022)Desai, Patel, Patel, Shah, Raval, and Ghosal]{desai2022identification}
B.~Desai, A.~Patel, V.~Patel, S.~Shah, M.~S. Raval, and R.~Ghosal.
\newblock Identification of free-ranging mugger crocodiles by applying deep learning methods on uav imagery.
\newblock \emph{Ecological Informatics}, 72:\penalty0 101874, 2022.
\bibitem[Jocher et~al.(2023)Jocher, Chaurasia, and Qiu]{Jocher_Ultralytics_YOLO_2023}
G.~Jocher, A.~Chaurasia, and J.~Qiu.
\newblock {Ultralytics YOLO}, Jan. 2023.
\newblock URL \url{https://github.com/ultralytics/ultralytics}.
\bibitem[Koger et~al.(2023)Koger, Deshpande, Kerby, Graving, Costelloe, and Couzin]{koger2023quantifying}
B.~Koger, A.~Deshpande, J.~T. Kerby, J.~M. Graving, B.~R. Costelloe, and I.~D. Couzin.
\newblock Quantifying the movement, behaviour and environmental context of group-living animals using drones and computer vision.
\newblock \emph{Journal of Animal Ecology}, 92\penalty0 (7):\penalty0 1357--1371, 2023.
\bibitem[Li et~al.(2022)Li, Erickson, and Xiong]{li2022individual}
G.~Li, G.~E. Erickson, and Y.~Xiong.
\newblock Individual beef cattle identification using muzzle images and deep learning techniques.
\newblock \emph{Animals}, 12\penalty0 (11):\penalty0 1453, 2022.
\bibitem[Li et~al.(2019)Li, Li, Tang, Qian, and Lin]{li2019atrw}
S.~Li, J.~Li, H.~Tang, R.~Qian, and W.~Lin.
\newblock Atrw: a benchmark for amur tiger re-identification in the wild.
\newblock \emph{arXiv preprint arXiv:1906.05586}, 2019.
\bibitem[Luiten et~al.(2021)Luiten, Osep, Dendorfer, Torr, Geiger, Leal-Taix{\'e}, and Leibe]{luiten2021hota}
J.~Luiten, A.~Osep, P.~Dendorfer, P.~Torr, A.~Geiger, L.~Leal-Taix{\'e}, and B.~Leibe.
\newblock Hota: A higher order metric for evaluating multi-object tracking.
\newblock \emph{International journal of computer vision}, 129:\penalty0 548--578, 2021.
\bibitem[Maeda et~al.(2021)Maeda, Ochi, Ringhofer, Sosa, Sueur, Hirata, and Yamamoto]{maeda2021aerial}
T.~Maeda, S.~Ochi, M.~Ringhofer, S.~Sosa, C.~Sueur, S.~Hirata, and S.~Yamamoto.
\newblock Aerial drone observations identified a multilevel society in feral horses.
\newblock \emph{Scientific Reports}, 11\penalty0 (1):\penalty0 71, 2021.
\bibitem[M{\"u}cher et~al.(2022)M{\"u}cher, Los, Franke, and Kamphuis]{mucher2022detection}
C.~M{\"u}cher, S.~Los, G.~Franke, and C.~Kamphuis.
\newblock Detection, identification and posture recognition of cattle with satellites, aerial photography and uavs using deep learning techniques.
\newblock \emph{International Journal of Remote Sensing}, 43\penalty0 (7):\penalty0 2377--2392, 2022.
\bibitem[Naik et~al.(2023)Naik, Chan, Yang, Delacoux, Couzin, Kano, and Nagy]{naik20233d}
H.~Naik, A.~H.~H. Chan, J.~Yang, M.~Delacoux, I.~D. Couzin, F.~Kano, and M.~Nagy.
\newblock 3d-pop-an automated annotation approach to facilitate markerless 2d-3d tracking of freely moving birds with marker-based motion capture.
\newblock In \emph{Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition}, pages 21274--21284, 2023.
\bibitem[Ozog{\'a}ny et~al.(2023)Ozog{\'a}ny, Kerekes, F{\"u}l{\"o}p, Barta, and Nagy]{ozogany2023fine}
K.~Ozog{\'a}ny, V.~Kerekes, A.~F{\"u}l{\"o}p, Z.~Barta, and M.~Nagy.
\newblock Fine-scale collective movements reveal present, past and future dynamics of a multilevel society in przewalski’s horses.
\newblock \emph{Nature Communications}, 14\penalty0 (1):\penalty0 5096, 2023.
\bibitem[Paulet et~al.(2024)Paulet, Molina, Beltzung, Suzumura, Yamamoto, and Sueur]{paulet2024deep}
J.~Paulet, A.~Molina, B.~Beltzung, T.~Suzumura, S.~Yamamoto, and C.~Sueur.
\newblock Deep learning for automatic facial detection and recognition in japanese macaques: illuminating social networks.
\newblock \emph{Primates}, pages 1--15, 2024.
\bibitem[Price et~al.(2023)Price, Khandelwal, Rubenstein, and Ahmad]{price2023framework}
E.~Price, P.~C. Khandelwal, D.~I. Rubenstein, and A.~Ahmad.
\newblock A framework for fast, large-scale, semi-automatic inference of animal behavior from monocular videos.
\newblock \emph{bioRxiv}, pages 2023--07, 2023.
\bibitem[Rathore et~al.(2023{\natexlab{a}})Rathore, Isvaran, and Guttal]{rathore2023lekking}
A.~Rathore, K.~Isvaran, and V.~Guttal.
\newblock Lekking as collective behaviour.
\newblock \emph{Philosophical Transactions of the Royal Society B}, 378\penalty0 (1874):\penalty0 20220066, 2023{\natexlab{a}}.
\bibitem[Rathore et~al.(2023{\natexlab{b}})Rathore, Sharma, Shah, Sharma, Torney, and Guttal]{rathore2023multi}
A.~Rathore, A.~Sharma, S.~Shah, N.~Sharma, C.~Torney, and V.~Guttal.
\newblock Multi-object tracking in heterogeneous environments (mothe) for animal video recordings.
\newblock \emph{PeerJ}, 11:\penalty0 e15573, 2023{\natexlab{b}}.
\bibitem[Romero-Ferrero et~al.(2019)Romero-Ferrero, Bergomi, Hinz, Heras, and De~Polavieja]{romero2019idtracker}
F.~Romero-Ferrero, M.~G. Bergomi, R.~C. Hinz, F.~J. Heras, and G.~G. De~Polavieja.
\newblock Idtracker. ai: tracking all individuals in small or large collectives of unmarked animals.
\newblock \emph{Nature methods}, 16\penalty0 (2):\penalty0 179--182, 2019.
\bibitem[Rosenthal et~al.(2015)Rosenthal, Twomey, Hartnett, Wu, and Couzin]{rosenthal2015revealing}
S.~B. Rosenthal, C.~R. Twomey, A.~T. Hartnett, H.~S. Wu, and I.~D. Couzin.
\newblock Revealing the hidden networks of interaction in mobile animal groups allows prediction of complex behavioral contagion.
\newblock \emph{Proceedings of the National Academy of Sciences}, 112\penalty0 (15):\penalty0 4690--4695, 2015.
\bibitem[Sampaio et~al.(2024)Sampaio, Sridhar, Francisco, Nagy, Sacchi, Strandburg-Peshkin, N{\"u}hrenberg, Rosa, Couzin, and Gingins]{sampaio2024multidimensional}
E.~Sampaio, V.~H. Sridhar, F.~A. Francisco, M.~Nagy, A.~Sacchi, A.~Strandburg-Peshkin, P.~N{\"u}hrenberg, R.~Rosa, I.~D. Couzin, and S.~Gingins.
\newblock Multidimensional social influence drives leadership and composition-dependent success in octopus--fish hunting groups.
\newblock \emph{Nature Ecology \& Evolution}, pages 1--13, 2024.
\bibitem[Schofield et~al.(2019)Schofield, Nagrani, Zisserman, Hayashi, Matsuzawa, Biro, and Carvalho]{schofield2019chimpanzee}
D.~Schofield, A.~Nagrani, A.~Zisserman, M.~Hayashi, T.~Matsuzawa, D.~Biro, and S.~Carvalho.
\newblock Chimpanzee face recognition from videos in the wild using deep learning.
\newblock \emph{Science advances}, 5\penalty0 (9):\penalty0 eaaw0736, 2019.
\bibitem[Shao et~al.(2020)Shao, Kawakami, Yoshihashi, You, Kawase, and Naemura]{shao2020cattle}
W.~Shao, R.~Kawakami, R.~Yoshihashi, S.~You, H.~Kawase, and T.~Naemura.
\newblock Cattle detection and counting in uav images based on convolutional neural networks.
\newblock \emph{International Journal of Remote Sensing}, 41\penalty0 (1):\penalty0 31--52, 2020.
\bibitem[Smith et~al.(2021)Smith, Napp, and Petersen]{smith2021imperfect}
M.~L. Smith, N.~Napp, and K.~H. Petersen.
\newblock Imperfect comb construction reveals the architectural abilities of honeybees.
\newblock \emph{Proceedings of the National Academy of Sciences}, 118\penalty0 (31):\penalty0 e2103605118, 2021.
\bibitem[Sosna et~al.(2019)Sosna, Twomey, Bak-Coleman, Poel, Daniels, Romanczuk, and Couzin]{sosna2019individual}
M.~M. Sosna, C.~R. Twomey, J.~Bak-Coleman, W.~Poel, B.~C. Daniels, P.~Romanczuk, and I.~D. Couzin.
\newblock Individual and collective encoding of risk in animal groups.
\newblock \emph{Proceedings of the National Academy of Sciences}, 116\penalty0 (41):\penalty0 20556--20561, 2019.
\bibitem[Sridhar et~al.(2021)Sridhar, Li, Gorbonos, Nagy, Schell, Sorochkin, Gov, and Couzin]{sridhar2021geometry}
V.~H. Sridhar, L.~Li, D.~Gorbonos, M.~Nagy, B.~R. Schell, T.~Sorochkin, N.~S. Gov, and I.~D. Couzin.
\newblock The geometry of decision-making in individuals and collectives.
\newblock \emph{Proceedings of the National Academy of Sciences}, 118\penalty0 (50):\penalty0 e2102157118, 2021.
\bibitem[Sridhar et~al.(2023)Sridhar, Davidson, Twomey, Sosna, Nagy, and Couzin]{sridhar2023inferring}
V.~H. Sridhar, J.~D. Davidson, C.~R. Twomey, M.~M. Sosna, M.~Nagy, and I.~D. Couzin.
\newblock Inferring social influence in animal groups across multiple timescales.
\newblock \emph{Philosophical Transactions of the Royal Society B}, 378\penalty0 (1874):\penalty0 20220062, 2023.
\bibitem[Torney et~al.(2018)Torney, Lamont, Debell, Angohiatok, Leclerc, and Berdahl]{torney2018inferring}
C.~J. Torney, M.~Lamont, L.~Debell, R.~J. Angohiatok, L.-M. Leclerc, and A.~M. Berdahl.
\newblock Inferring the rules of social interaction in migrating caribou.
\newblock \emph{Philosophical Transactions of the Royal Society B: Biological Sciences}, 373\penalty0 (1746):\penalty0 20170385, 2018.
\bibitem[Tuia et~al.(2022)Tuia, Kellenberger, Beery, Costelloe, Zuffi, Risse, Mathis, Mathis, Van~Langevelde, Burghardt, et~al.]{tuia2022perspectives}
D.~Tuia, B.~Kellenberger, S.~Beery, B.~R. Costelloe, S.~Zuffi, B.~Risse, A.~Mathis, M.~W. Mathis, F.~Van~Langevelde, T.~Burghardt, et~al.
\newblock Perspectives in machine learning for wildlife conservation.
\newblock \emph{Nature communications}, 13\penalty0 (1):\penalty0 1--15, 2022.
\bibitem[Wahltinez and Wahltinez(2024)]{wahltinez2024open}
O.~Wahltinez and S.~J. Wahltinez.
\newblock An open-source general purpose machine learning framework for individual animal re-identification using few-shot learning.
\newblock \emph{Methods in Ecology and Evolution}, 15\penalty0 (2):\penalty0 373--387, 2024.
\bibitem[Waldmann et~al.(2024)Waldmann, Chan, Naik, Nagy, Couzin, Deussen, Goldluecke, and Kano]{waldmann20243d}
U.~Waldmann, A.~H.~H. Chan, H.~Naik, M.~Nagy, I.~D. Couzin, O.~Deussen, B.~Goldluecke, and F.~Kano.
\newblock 3d-muppet: 3d multi-pigeon pose estimation and tracking.
\newblock \emph{International Journal of Computer Vision}, pages 1--18, 2024.
\bibitem[Walter and Couzin(2021)]{walter2021trex}
T.~Walter and I.~D. Couzin.
\newblock Trex, a fast multi-animal tracking system with markerless identification, and 2d estimation of posture and visual fields.
\newblock \emph{Elife}, 10:\penalty0 e64000, 2021.
\bibitem[Wang et~al.(2019)Wang, Jiao, Yang, Yang, Zhang, Wei, and Zhang]{wang2019vehicle}
P.~Wang, B.~Jiao, L.~Yang, Y.~Yang, S.~Zhang, W.~Wei, and Y.~Zhang.
\newblock Vehicle re-identification in aerial imagery: Dataset and approach.
\newblock In \emph{Proceedings of the IEEE/CVF International Conference on Computer Vision}, pages 460--469, 2019.
\bibitem[Weinstein et~al.(2022)Weinstein, Garner, Saccomanno, Steinkraus, Ortega, Brush, Yenni, McKellar, Converse, Lippitt, et~al.]{weinstein2022general}
B.~G. Weinstein, L.~Garner, V.~R. Saccomanno, A.~Steinkraus, A.~Ortega, K.~Brush, G.~Yenni, A.~E. McKellar, R.~Converse, C.~D. Lippitt, et~al.
\newblock A general deep learning model for bird detection in high-resolution airborne imagery.
\newblock \emph{Ecological Applications}, 32\penalty0 (8):\penalty0 e2694, 2022.
\bibitem[Wu et~al.(2019)Wu, Kirillov, Massa, Lo, and Girshick]{wu2019detectron2}
Y.~Wu, A.~Kirillov, F.~Massa, W.-Y. Lo, and R.~Girshick.
\newblock Detectron2.
\newblock \url{https://github.com/facebookresearch/detectron2}, 2019.
\bibitem[Zhang et~al.(2023)Zhang, Gao, Xiao, and Fan]{zhang2023animaltrack}
L.~Zhang, J.~Gao, Z.~Xiao, and H.~Fan.
\newblock Animaltrack: A benchmark for multi-animal tracking in the wild.
\newblock \emph{International Journal of Computer Vision}, 131\penalty0 (2):\penalty0 496--513, 2023.
\bibitem[Zhang et~al.(2021)Zhang, Sun, Jiang, Yu, Yuan, Luo, Liu, and Wang]{zhang2021bytetrack}
Y.~Zhang, P.~Sun, Y.~Jiang, D.~Yu, Z.~Yuan, P.~Luo, W.~Liu, and X.~Wang.
\newblock Bytetrack: Multi-object tracking by associating every detection box. corr abs/2110.06864 (2021).
\newblock \emph{URL: https://arxiv. org/abs/2110}, 6864, 2021.
\bibitem[Zhu et~al.(2021)Zhu, Wen, Du, Bian, Fan, Hu, and Ling]{zhu2021detection}
P.~Zhu, L.~Wen, D.~Du, X.~Bian, H.~Fan, Q.~Hu, and H.~Ling.
\newblock Detection and tracking meet drones challenge.
\newblock \emph{IEEE Transactions on Pattern Analysis and Machine Intelligence}, 44\penalty0 (11):\penalty0 7380--7399, 2021.
\end{thebibliography}