Multi-Task Optimization over Networks of Tasks
Extracted source text · arXiv 2604.21991
\documentclass[runningheads]{llncs}
%
\usepackage[T1]{fontenc}
% T1 fonts will be used to generate the final print and online PDFs,
% so please use T1 fonts in your manuscript whenever possible.
% Other font encondings may result in incorrect characters.
%
\usepackage{graphicx}
\usepackage{float}
\usepackage{amsmath}
\usepackage{amssymb}
\usepackage{booktabs}
\usepackage{algorithm, algpseudocode}
\usepackage{url}
\usepackage{caption}
\usepackage{subcaption}
\usepackage{xcolor}
\usepackage{paralist}
\usepackage{cite}
\usepackage{hyperref}
% Short handles for the MONET GitHub URLs (used in footnotes).
\newcommand{\MONETrepo}{https://github.com/ju2ez/MONET/tree/PPSN-MONET-introduction}
\newcommand{\MONETsupp}{https://github.com/ju2ez/MONET/tree/PPSN-MONET-introduction/supplementary}
\begin{document}
%
\title{Multi-Task Optimization over Networks of Tasks}
%
%\titlerunning{MONET}
% If the paper title is too long for the running head, you can set
% an abbreviated paper title here
%
\author{Julian Hatzky\inst{1} \and
Thomas Bartz-Beielstein\inst{2} \and
A. E. Eiben\inst{1} \and
Anil Yaman\inst{1}}
%
\authorrunning{J. Hatzky et al.}
% First names are abbreviated in the running head.
% If there are more than two authors, 'et al.' is used.
%
\institute{Vrije Universiteit Amsterdam, Amsterdam, The Netherlands\\
\email{\{j.hatzky, a.e.eiben, a.yaman\}@vu.nl} \and
TH Köln, Cologne, Germany\\
\email{thomas.bartz-beielstein@th-koeln.de}}
%
\maketitle % typeset the header of the contribution
%
\begin{abstract}
%Multi-task optimization algorithms aim to find high-performing solutions for many tasks simultaneously and in parallel. Among the algorithms proposed, population based variants are not scalable to a high number of tasks. Alternatively,
%Most algorithms were not scalable (population based ones), and the ones that are scalable are archive based (MAP-Elites variants) do not exploit the task structure.
%Here, we propose a novel algorithm that can exploit the task structure by representing tasks as nodes in a graph, and establishing connectivity between nodes to utilize knowledge transfer
Multi-task optimization is a powerful approach for solving a large number of tasks in parallel.
However, existing algorithms face distinct limitations:
Population-based methods scale poorly and remain underexplored for large task sets.
Approaches that do scale beyond a thousand tasks are mostly MAP-Elites variants and rely on a fixed, discretized archive that disregards the topology of the task space.
We introduce MONET (Multi-Task Optimization over Networks of Tasks), a multi-task optimization algorithm that models the task space as a graph: tasks are nodes, and edges connect tasks in the task parameter space. This representation enables knowledge transfer between tasks and remains tractable for high-dimensional problems while exploiting the topology of the task space. MONET combines \emph{social learning}, which generates candidates from neighboring nodes via crossover, with \emph{individual learning}, which refines a node's own solution independently via mutation. We evaluate MONET on four domains (archery, arm, and cartpole with 5,000 tasks each; hexapod with 2,000 tasks) and show that it matches or exceeds the performance of existing MAP--Elites-based baselines across all four domains.
\keywords{Multi-task Optimization \and Multi-Task MAP-Elites \and Innovation Networks \and Social Learning \and Graphs}
\end{abstract}
%multitask-optimization papers: https://github.com/xiaofangxd/Multitasking-Optimization
\section{Introduction}\label{lab:intro}
To maintain reliable performance, real-world embodied agents must remain operational even when their configurations change. For example, in scenarios where a legged robot loses a limb~\cite{MAP_Elites_NaturePaper,bongard2006resilient} or a modular robot reconfigures itself to navigate a narrow passage~\cite{yim2007modular,vandiggelen2024modelfree,samarakoon2022metaheuristic}, they are required to maintain their performance. In such cases, the ability to adapt an existing control strategy instead of relearning a new strategy for the new configuration from scratch can prove to be more efficient. It is possible to achieve this by employing the multi-task optimization approach~\cite{gupta2015multifactorial}, where each configuration variant (e.g. morphology) is treated as a distinct task, and solutions (e.g. controller) are pre-computed for all of them, so that when the configurations change, an appropriate controller is already available or is faster to adapt~\cite{DBLP:journals/corr/MouretC15,chatzilygeroudis2021qd}. In other words, given a parameterized family of tasks, the multi-task optimization approach aims to find a high-quality solution for each task simultaneously and in parallel~\cite{gupta2015multifactorial}.
%aims to achieve this by treating (e.g., a morphology) as a distinct task and pre-computing a repertoire of high-performing solutions (i.e., controllers) across all of them, so that when the configurations change, an appropriate controller is already available or is faster to adapt\textcolor{red}{[cite]}. This is the setting of multi-task optimization~\textcolor{red}{[cite]}: given a parameterized family of tasks, find a high-quality solution for each task simultaneously.
\begin{figure}
\centering
\includegraphics[width=\textwidth]{figs/figures_monet_graph.pdf}
\caption{Illustration of the MONET task network structure. Each node $V_i = \boldsymbol{\tau}_i$ represents a task, labelled by its current best solution $\mathcal{A}[\boldsymbol{\tau}_i] \in \Theta$. (a) Task network with the individual and social learning strategies that operate on it. (b) Task network and solution qualities at $500{,}000$ evaluations ($50\%$ of the total).}
\label{fig:teaser}
\end{figure}
Existing multi-task optimization algorithms face a fundamental scalability--structure tradeoff. Population-based multi-task evolutionary methods~\cite{huynhthithanhEnsembleMultifactorialEvolutionBiasedSkillFactorInheritanceManyTaskOptimization2023,wangSolvingMultitaskOptimizationProblemsAdaptiveKnowledgeTransferAnomalyDetection2022} maintain a separate population per task and quickly become impractical as the number of tasks grows into the thousands and beyond. Quality-diversity (QD) algorithms and MAP-Elites~\cite{MAP_Elites_NaturePaper,DBLP:journals/corr/MouretC15} in particular, bypass this by replacing per-task populations with a single structured archive. MAP-Elites has been extended to multi-task settings by Multi-Task MAP-Elites (MT-ME)~\cite{mouretQualitydiversitymultitaskoptimization2020}, which solves a fixed set of tasks via a globally shared archive, and Parametric-Task MAP-Elites (PT-ME)~\cite{anneParametricTaskMAPElites2024}, which extends this to continuous task spaces. Both have demonstrated strong performance on problems with thousands of tasks.
However, these archive-based methods exhibit two intertwined limitations. First, the archive is a \emph{fixed discretized grid} over the task or feature space: with $c$ bins per dimension and a $d$-dimensional task space its size grows as $\mathcal{O}(c^d)$, so the representation becomes infeasible already for moderate $d$. Second, the archive is largely \emph{topologically flat}: in MT-ME any solution can serve as a parent for any task. PT-ME partially addresses this through a local linear regression operator that exploits Delaunay triangulation~\cite{Delaunay1928} between tasks to calculate an adjacency matrix. Even so, this locality is confined to one of two variation paths, and the global Simulated Binary Crossover (SBX) operator~\cite{SBX} still draws parents without regard for task similarity. Neither method makes the task-space topology a first-class structural element that governs all recombination.
We introduce MONET (Multi-Task Optimization over Networks of Tasks), a multi-task optimization algorithm that resolves both limitations by replacing the flat archive with an explicit graph over tasks (Figure~\ref{fig:teaser}): each node in the vertex set $\mathcal{V}$ represents a task and stores its current best solution, while bidirectional edges in $\mathcal{E}$ connect tasks to enable knowledge transfer (e.g., through crossover or direct copy). Unlike a grid, this representation does not depend on a discretization of the task space and scales as $\mathcal{O}(|\mathcal{V}|\,k)$, where $k$ is the number of neighbors per node, remaining tractable as the task space dimensionality grows. Unlike a flat archive, the graph encodes topology directly: every node has a well-defined neighborhood that the search can exploit. The evolutionary process operates on this graph through two complementary mechanisms. \emph{Individual learning} improves a solution in isolation through mutation, exploring the local fitness landscape of a single task. \emph{Social learning} transfers information between neighboring nodes in variable ratios by selecting parent solutions from neighboring tasks and generating offspring through crossover. This separation is inspired by cultural evolution, where individual innovation and the social spread of useful innovations through a population jointly drive adaptation~\cite{rendell2010socialLearning,yaman2021distributed}.
The balance between individual and social learning, the strategy to select a node (random or best fit), the size of the neighborhood, and the type of neighborhood (closest neighbors, distance-proportional sampling, or random selection) are key design choices that shape how information flows through the graph. A small neighborhood with strong social learning concentrates exploitation among very similar tasks, while a larger neighborhood or weaker social learning permits broader exploration. Understanding which of these choices matters and whether a single robust configuration exists in different problem domains is essential for the practical application of MONET.
To this end, we conducted a hyperparameter study in four environments: archery, arm, and cartpole with 5,000 tasks each, and hexapod with 2,000 tasks. We show that MONET outperforms both MT-ME and PT-ME across all four environments, demonstrating that exploiting task-space structure through a network of task neighborhoods is an effective approach to multi-task optimization at scale.
Our contributions are:
\begin{compactenum}
\item A unified formal taxonomy of MAP-Elites variants for multi-task optimization that exposes the structural modifications distinguishing each variant and motivates MONET (Section~\ref{lab:related}).
\item MONET, a graph-based multi-task optimizer that makes task-space neighborhoods an explicit element of recombination and combines localized social and individual learning (Section~\ref{lab:methods}).
\item A systematic hyperparameter study across four task domains (archery, arm, cartpole, and hexapod) analyzing the effects of neighborhood type, neighborhood size, and the social/individual balance (Sections~\ref{lab:experimental} and~\ref{lab:results}).
\item Empirical evidence that a single MONET configuration generalizes across all four domains and matches or outperforms MT-ME and PT-ME, showing task-space topology to be a valuable and underexploited source of information in multi-task optimization (Sections~\ref{lab:results} and~\ref{lab:discussion}).
\end{compactenum}
\input{01_relatedWorks}
\input{02_methods}
\input{03_experimentalSetup}
\input{04_results}
\input{05_discussion}
\bibliographystyle{splncs04}
\bibliography{refs}
\appendix
\input{07_supplementaryAppend}
\end{document}
\section{Related Works}\label{lab:related}
Evolutionary multi-task optimization has become a highly active area of research within the computational intelligence community \cite{gupta2015multifactorial,EMT_6_applications,multi_task_survey,osaba2021evolutionarymultitaskoptimizationmethodological}.
To concretize the description, we specify the problem setting as follows.
Consider a set of $n$ tasks, each described by a parameter vector $\boldsymbol{\tau}_i \in \mathcal{T}$, for which we want to find the corresponding solutions $\boldsymbol{\theta}_1, \ldots, \boldsymbol{\theta}_n \in \Theta$ simultaneously. The goal of multi-task optimization is to find, for every task $\boldsymbol{\tau} \in \mathcal{T}$, an optimal solution as:
\begin{equation}
\boldsymbol{\theta}^{*}_{\boldsymbol{\tau}} = \arg\max_{\boldsymbol{\theta} \in \Theta} f(\boldsymbol{\theta}, \boldsymbol{\tau}),
\label{eq:multitask}
\end{equation}
where $\Theta$ is the solution space, $\mathcal{T}$ the task parameter space and $f: \Theta \times \mathcal{T} \rightarrow \mathbb{R}$ the fitness function.
Existing methodologies generally fall into two categories: \emph{implicit transfer}, which utilizes a single, unified search space for all tasks, and \emph{explicit transfer}, which operates across distinct solution spaces. Although implicit transfer is relatively straightforward to implement, its application is largely restricted to homogeneous environments. In contrast, explicit transfer accommodates heterogeneous tasks, though determining the optimal knowledge to exchange between distinct spaces remains a complex challenge. To facilitate this transfer, contemporary frameworks have leveraged probabilistic search distributions, search direction vectors, higher-order heuristics, and surrogate models. A representative example is the Multi-Factorial Evolutionary Algorithm II (MFEA-II)~\cite{bali2020mfea2}, which learns an online inter-task transfer parameter to modulate the probabilistic crossover between tasks and thereby suppress negative transfer during multifactorial search.
Historically, the application of these methods has been limited to optimizing a small number of tasks concurrently (typically between two and ten). As catalogued in the evolutionary multi-tasking (EMT) survey of~\cite{osaba2021evolutionarymultitaskoptimizationmethodological}, examples include optimizing three robot morphologies via \emph{neuroevolution}~\cite{bali2020mfea2}, two symbolic regression datasets using genetic programming~\cite{EMT_symbolic_regression_with_GP}, ten numerical functions for software testing~\cite{EMT_software_testing_multifactorial}, multi-UAV path planning across three tasks~\cite{EMT_UAV_path_planning}, power dispatch optimization for two bus systems~\cite{EMT_complex_design_electric_power_dispatch}, logistics planning for four package delivery problems~\cite{EMT_logistics_planning_combinatorial_optimzation}, and the design of three diode models~\cite{EMT_complex_design_photovoltaic_models}. Similar multi-task formulations exist in other fields, such as Bayesian optimization~\cite{Multi_task_bayesian_optimization}, which is frequently used to tune machine learning models across multiple datasets. However, because Bayesian approaches rely on Gaussian processes that scale cubically\footnote{In addition, the approaches such as Nyström approximation have limitations.}, they are restricted to computationally expensive fitness functions and struggle to scale beyond a thousand evaluations~\cite{Bayesian-Optimization_review}.
To address these scalability limits, recent works in evolutionary many-task optimization have successfully tackled increasingly larger problem sets, scaling to 30, 50, 500, and up to 2,000 tasks simultaneously. Aligning with this trajectory toward large-scale optimization. %, our approach builds upon the foundational principles of MT-ME and PT-ME, two algorithms demonstrated to scale efficiently to 5,000 concurrent tasks.
In order to show the connection of the proposed method to the existing literature, we first introduce a unified notation framework for the whole MAP-Elites algorithm family, enabling each multi-task variant to be presented as a modification of its underlying formulation\footnote{We adopt the maximization convention throughout; a minimization problem with cost $c$ is handled by setting $f = -c$.}. The same base is then extended once more in Section~\ref{lab:methods} when MONET is defined.
\begin{definition}[MAP-Elites]\label{def:mapelites}
Let $\Theta$ denote a solution space, $f : \Theta \to \mathbb{R}$ a fitness function to be \emph{maximized}, $\mathcal{B}$ a behavior space, and $\mathbf{b} : \Theta \to \mathcal{B}$ a behavioral descriptor that maps a solution to its behavior. Fix a finite discretization of $\mathcal{B}$ into cells. MAP-Elites~\cite{MAP_Elites_NaturePaper,DBLP:journals/corr/MouretC15} maintains an archive $\mathcal{A}$\footnote{We use uppercase calligraphic $\mathcal{A}$ because the archive is primarily a data structure that is \emph{treated as} a partial function; this convention is standard in the MAP-Elites literature and consistent with the use of uppercase for structured objects such as operators or transforms.}, treated as a partial function from cells to $\Theta$, initialized empty and updated by the following elitist rule: given a candidate $\boldsymbol{\theta} \in \Theta$, insert $\boldsymbol{\theta}$ into the cell containing $\mathbf{b}(\boldsymbol{\theta})$ iff that cell is empty or $f(\boldsymbol{\theta}) > f(\mathcal{A}[\mathbf{b}(\boldsymbol{\theta})])$. The run returns $\mathcal{A}$, which trades objective value against behavioral coverage rather than optimizing $f$ alone.
\end{definition}
Figure~\ref{fig:commutative} summarizes the mappings of Definition~\ref{def:mapelites} as a commutative diagram.\footnote{ Note that Definition~\ref{def:mapelites} uses a single-task fitness $f : \Theta \to \mathbb{R}$; the multi-task extensions below lift this to $f : \Theta \times \mathcal{T} \to \mathbb{R}$ by conditioning on a task parameter $\boldsymbol{\tau} \in \mathcal{T}$, recovering the problem statement of Eq.~\eqref{eq:multitask}.}
Now we are ready to introduce relevant multi-task variants as extensions of this definition: each paragraph names exactly what is changed and leaves the rest of the definition in place.
\begin{figure}[t]
\centering
\begin{subfigure}[t]{0.48\textwidth}
\centering
\includegraphics[width=\textwidth]{figs/mapelites_commutative.pdf}
\caption{MAP-Elites (Definition~\ref{def:mapelites}). Solid blue: behavioral descriptor $\mathbf{b} : \Theta \to \mathcal{B}$; solid red: fitness $f : \Theta \to \mathbb{R}$; dashed green: archive $\mathcal{A} : \mathcal{B} \rightharpoonup \Theta$ and composed map $f \circ \mathcal{A} : \mathcal{B} \to \mathbb{R}$.}
\label{fig:commutative}
\end{subfigure}\hfill
\begin{subfigure}[t]{0.48\textwidth}
\centering
\includegraphics[width=\textwidth]{figs/monet_commutative.pdf}
\caption{MONET (Definition~\ref{def:monet}). The behavioral descriptor $\mathbf{b}$ is absent; solid red: fitness $f(\cdot,\boldsymbol{\tau}) : \Theta \times \mathcal{T} \to \mathbb{R}$; dashed green: archive $\mathcal{A} : \mathcal{T} \to \Theta$ and $f \circ \mathcal{A} : \mathcal{T} \to \mathbb{R}$; solid blue: graph construction via task distance $d$; dashed blue: neighborhood lookup $\mathrm{Nb}(\boldsymbol{\tau})$.}
\label{fig:monet_commutative}
\end{subfigure}
\caption{Commutative diagrams comparing MAP-Elites and MONET.}
\label{fig:commutative_combined}
\end{figure}
\noindent\textbf{Multi-Task MAP-Elites (MT-ME)~\cite{mouretQualitydiversitymultitaskoptimization2020}.} MT-ME extends Definition~\ref{def:mapelites} by \emph{replacing} the behavioral descriptor $\mathbf{b} : \Theta \to \mathcal{B}$ with the task parameter itself. A finite set of tasks $\{\boldsymbol{\tau}_1, \ldots, \boldsymbol{\tau}_n\} \subset \mathcal{T}$ is fixed in advance, the archive becomes $\mathcal{A} : \mathcal{T} \to \Theta$ indexed directly by task, and the fitness lifts to $f : \Theta \times \mathcal{T} \to \mathbb{R}$, matching the problem statement of Eq.~\eqref{eq:multitask}. Recombination draws parents globally from the entire task archive, so information is shared across all tasks regardless of their similarity. MT-ME uses the iso-line-dd variation operator~\cite{Hypervolume_MAP-Elites}, which exploits the observation that elites tend to cluster in a compact ``elite hypervolume'' in genotype space: given two parent elites $\boldsymbol{\theta}_i$ and $\boldsymbol{\theta}_j$, a candidate is generated by adding isotropic Gaussian noise and a directional component along the line connecting the parents, thereby biasing search toward the region of genotype space where elites concentrate.
Mouret and Maguire~\cite{mouretQualitydiversitymultitaskoptimization2020} evaluated MT-ME on arm and hexapod (Figure \ref{fig:benchmark-suite}), comparing against CMA-ES run independently per task, MAP-Elites with random task assignment, and MAP-Elites evaluated on all tasks simultaneously. MT-ME outperformed all baselines on both domains.
\noindent\textbf{Multi-Task Multi-Behavior MAP-Elites (MTMB-ME)~\cite{MTMB-ME}.} MTMB-ME extends MT-ME by \emph{restoring} the behavioral descriptor from Definition~\ref{def:mapelites} on top of the task index, yielding a two-level archive $\mathcal{A} : \mathcal{T} \times \mathcal{B} \to \Theta$ indexed jointly by task and behavior. Task-specific adaptation and behavioral diversity are therefore maintained simultaneously within a single archive. Anne and Mouret~\cite{MTMB-ME} applied MTMB-ME to a humanoid robot fault-recovery scenario, identifying successful contact positions on a wall that help the robot regain balance after a leg fault, and outperformed random search, grid search, and standard MAP-Elites both in the number of tasks solved and in the number of solutions found.
\noindent\textbf{Parametric-Task MAP-Elites (PT-ME)~\cite{anneParametricTaskMAPElites2024}.} PT-ME extends MT-ME to \emph{continuous} task spaces. Instead of fixing a finite set of tasks in advance, PT-ME draws a fresh task $\boldsymbol{\tau} \sim \mathcal{T}$ at every iteration and stores, for each drawn task, the best solution encountered so far, so the archive becomes a dense dataset covering the entire task space. PT-ME alternates between two variation operators at equal probability. The first is SBX~\cite{SBX} with a bandit-tuned tournament that biases task selection toward tasks close to the parents, replacing the iso-line-dd operator of MT-ME. The second, and more distinctive, operator is a \emph{local linear regression}: PT-ME partitions the task space into cells via Centroidal Voronoi Tessellation (CVT)~\cite{CVT,CVT-MAP-Elites}, precomputes a $k$-dimensional tree ($k$-d tree)~\cite{bentley1975multidimensional} for fast cell lookup and a Delaunay triangulation~\cite{Delaunay1928} for adjacency, then fits a linear least-squares model $M = (\mathbf{T}^\top \mathbf{T})^{-1}\mathbf{T}^\top \mathbf{X}$ to the elites in adjacent cells, where $\mathbf{T}$ stacks the task vectors of the adjacent cells and $\mathbf{X}$ the corresponding solution vectors; a candidate is then generated as $\boldsymbol{\theta} = M \cdot \boldsymbol{\tau} + \sigma_{\mathrm{reg}} \cdot \mathcal{N}(0, \mathrm{var}(\mathbf{X}))$. This regression operator is notable because it explicitly exploits task-space locality: the Delaunay triangulation defines a geometric neighborhood over $\mathcal{T}$, and the linear model builds a local approximation of the task-to-solution mapping from that neighborhood. On archery adn arm PT-ME yields improved performance over standard MAP-Elites and Proximal Policy Optimization (PPO)~\cite{PPO}.
\noindent\textbf{MAP-Elites with Knowledge Transfer (MMKT)~\cite{9870374}.} This variant extends MT-ME by \emph{partitioning} the task set into groups $\{\mathcal{T}_1, \ldots, \mathcal{T}_m\}$ obtained by applying $k$-means clustering to the task parameters, and then promoting knowledge transfer between tasks through linear combinations of solutions drawn from the same group. In this preliminary study, MMKT outperforms MT-ME on the arm task.
\noindent\textbf{Adaptive Group-based Collaborative Optimization (AGCO)~\cite{houGroupBasedManyTaskCollaborativeOptimizationFrameworkEvolutionaryRobotsDesign2025}.} This framework extends the group-based view further with a two-stage transfer protocol. An \emph{inter-group knowledge separation} stage employs an adaptive selection mechanism for crossover operators to control information exchange between groups, and an \emph{intra-group knowledge reunion} stage leverages Thompson sampling to weight within-group recombination so that target tasks assimilate insights from multiple in-group sources according to their evolutionary preferences.
Most of the variants above treat the archive as fully connected: every elite can serve as a parent for any cell regardless of the relationship between the corresponding tasks. PT-ME is a partial exception: its regression operator builds a local linear model from Delaunay-adjacent cells, thereby exploiting geometric locality in $\mathcal{T}$ for candidate generation. However, this locality is confined to one of two variation operators (applied with 50\% probability), the adjacency structure is a by-product of the CVT tessellation rather than an explicit design element, and the SBX path still draws parents globally. An alternative that makes locality a first-class principle is familiar from \emph{cellular evolutionary algorithms} (cEAs)~\cite{alba2008cellular}, which organize the population into local neighborhoods and restrict recombination accordingly, so that information propagates through the population only via those neighborhoods. MONET (Section~\ref{lab:methods}) follows the same intuition but imposes the graph over tasks rather than over individuals: it extends MT-ME by replacing the implicit fully-connected archive $\mathcal{A} : \mathcal{T} \to \Theta$ with an explicit graph structure over $\mathcal{T}$, inducing neighborhoods from task-space proximity.
\section{MONET}\label{lab:methods}
Building on the taxonomy introduced in Section~\ref{lab:related}, we now introduce MONET. Figure~\ref{fig:teaser} illustrates the task-graph representation at the core of MONET: nodes are tasks, edges connect neighboring tasks in the task parameter space, and each node stores the best solution found so far.
\begin{definition}[MONET]\label{def:monet}
Let $\{\boldsymbol{\tau}_1,\ldots,\boldsymbol{\tau}_n\}\subset\mathcal{T}$ be a finite task set, $f:\Theta\times\mathcal{T}\to\mathbb{R}$ a task-parameterized fitness function, and $d:\mathcal{T}\times\mathcal{T}\to\mathbb{R}_{\ge 0}$ a task distance. MONET represents the optimization state as a labeled graph $\mathcal{G}=(\mathcal{V},\mathcal{E},\mathcal{A})$ whose nodes $\mathcal{V}=\{\boldsymbol{\tau}_1,\ldots,\boldsymbol{\tau}_n\}$ are the tasks themselves, whose edges $\mathcal{E}=\{(\boldsymbol{\tau}_i,\boldsymbol{\tau}_j):\boldsymbol{\tau}_j\text{ is among the }N \text{ nearest neighbors of } \boldsymbol{\tau}_i\text{ under }d\}$ encode task-space proximity for a user-chosen neighborhood size $N$, and whose labeling $\mathcal{A}:\mathcal{V}\to\Theta$ stores the best solution found so far for each task. Candidate solutions are admitted by the elitist rule: a candidate $\boldsymbol{\theta}$ generated for task $\boldsymbol{\tau}_i$ replaces $\mathcal{A}[\boldsymbol{\tau}_i]$ iff $f(\boldsymbol{\theta},\boldsymbol{\tau}_i) \ge f(\mathcal{A}[\boldsymbol{\tau}_i],\boldsymbol{\tau}_i)$.
\end{definition}
Figure~\ref{fig:monet_commutative} summarizes the mappings of Definition~\ref{def:monet}. Compared with the base MAP-Elites diagram (Figure~\ref{fig:commutative}), the behavioral descriptor $\mathbf{b}$ is absent and the archive is indexed directly by task; the fitness is conditioned on a task parameter; and the bottom row shows the graph construction from the task distance $d$ and the neighborhood lookup $\mathrm{Nb}(\boldsymbol{\tau})$ that mediates social learning.
Candidates are generated by one of two complementary mechanisms, selected at each step with probability $p_{\text{ind}}$. \emph{Individual learning} mutates the current label $\mathcal{A}[\boldsymbol{\tau}_i]$ of a focal task, exploring the local fitness landscape of $\boldsymbol{\tau}_i$ in isolation. \emph{Social learning} draws a neighbor $\boldsymbol{\tau}_j$ from the graph neighborhood of $\boldsymbol{\tau}_i$ and recombines $\mathcal{A}[\boldsymbol{\tau}_i]$ with $\mathcal{A}[\boldsymbol{\tau}_j]$, transferring information between similar tasks along the edges of $\mathcal{G}$. Empty nodes are initialized by individual learning with a uniform random solution, or by social learning by copying from a non-empty neighbor (with a uniform random fallback if no such neighbor exists); social learning is skipped at a step where the chosen neighbor is itself empty.
In this work, we use the Euclidean distance in $\mathcal{T}$ as the task distance $d$, and set the neighborhood size $N$ (the number of nearest neighbors retained per task, Def.~\ref{def:monet}) to a small fraction of $|\mathcal{V}|$ (see Section~\ref{lab:experimental}), so that $\mathcal{G}$ is a sparse, partially connected graph whose density is controlled by the chosen neighborhood size. Algorithm~\ref{alg:monet} provides the pseudocode of MONET and the code is publicly accessible~\cite{monet_code}.
\begin{algorithm}[!t]
\caption{MONET}\label{alg:monet}
\footnotesize
\setlength{\textfloatsep}{6pt plus 1pt minus 1pt}
\begin{algorithmic}[1]
\Require Task set $T=\{\boldsymbol{\tau}_1,\dots,\boldsymbol{\tau}_n\}$, fitness function $f$, task distance $d$
\Require Budget $E$, init fraction $K_{\text{init}}$, neighborhood size $N$, selection strategy \Call{Select}{$\cdot$}, probability $p_{\text{ind}}$
\Ensure Archive $\mathcal{A}$ mapping each task to its best solution
\State $\mathcal{A}[\boldsymbol{\tau}] \gets \emptyset$ for all $\boldsymbol{\tau} \in T$ \Comment{empty network}
\For{each $\boldsymbol{\tau}_i \in T$} \Comment{build $k$-NN task graph}
\State $\mathrm{Nb}(\boldsymbol{\tau}_i) \gets N$ nearest tasks under $d$
\EndFor
\For{$i \gets 1$ \textbf{to} $K_{\text{init}}$} \Comment{random initialization}
\State $\boldsymbol{\tau} \gets \Call{RandomTask}{T}$
\State $\boldsymbol{\theta} \sim \mathcal{U}(p_{\min}, p_{\max})$
\State $f_x \gets f(\boldsymbol{\theta}, \boldsymbol{\tau})$
\If{$\mathcal{A}[\boldsymbol{\tau}] = \emptyset$ \textbf{or} $f_x \geq \mathcal{A}[\boldsymbol{\tau}].\text{fitness}$} \Comment{elitist update}
\State $\mathcal{A}[\boldsymbol{\tau}] \gets (\boldsymbol{\theta}, f_x)$
\EndIf
\EndFor
\State $e \gets K_{\text{init}}$ \Comment{main loop}
\While{$e < E$}
\State $\boldsymbol{\tau} \gets \Call{RandomTask}{T}$ \Comment{sample focal task}
\If{$\mathrm{rand}() < p_{\text{ind}}$} \Comment{individual learning}
\State \Call{Mutation}{$\boldsymbol{\tau}, \mathcal{A}$} \Comment{\Call{IndividualLearning}{$\boldsymbol{\tau}, \mathcal{A}$}}
\State $e \gets e + 1$
\Else \Comment{social learning}
\State $\boldsymbol{\tau}_j \gets \Call{Select}{\mathrm{Nb}(\boldsymbol{\tau})}$ \Comment{pick one neighbor}
\If{$\boldsymbol{\tau}_j \neq \text{nil}$} \Comment{skip if no graph neighbor exists}
\State \Call{SBX}{$\boldsymbol{\tau}, \boldsymbol{\tau}_j, \mathcal{A}$} \Comment{\Call{SocialLearning}{$\boldsymbol{\tau}, \boldsymbol{\tau}_j, \mathcal{A}$}}
\State $e \gets e + 1$
\EndIf
\EndIf
\EndWhile
\State \Return $\mathcal{A}$
\end{algorithmic}
\end{algorithm}
%\begin{enumerate}
% \item Modular evolution
% \item ANN evolution - direct, NEAT, modular?
% \item Tree structured
% \item Knowledge transfer
%\end{enumerate}
%incoporate also reinforcement learning?
%possible tasks:
%control: cartpole, robotic arm, ...
%bipedal walking: controllers for different bodies, controllers and different bodies for different environments
%morphology evolution: bodies for different environments
%multi-agent: different task/problem designs
%plasticity evolution:
%social norms
%environment analysis
\section{Experimental Setup}\label{lab:experimental}
We compare MONET against PT-ME and MT-ME across four task environments: archery, arm, cartpole, and hexapod (Figure \ref{fig:benchmark-suite}).
PT-ME samples tasks continuously from $\mathcal{T}$ rather than operating over a fixed task set: it partitions $\mathcal{T}$ into $n$ cells via Centroidal Voronoi Tessellation (CVT)~\cite{CVT,CVT-MAP-Elites} and, by the end of its run, each cell holds the highest-fitness task-solution pair it has sampled. To place the three algorithms on the same footing, we take these $n$ cell elites ($n=5000$ for archery, arm, and cartpole, and $n=2000$ for hexapod) as the fixed task set on which MT-ME and MONET are then run. This removes task selection as a confound in the cross-algorithm comparison. One asymmetry remains: under this protocol each PT-ME task is evaluated essentially once (the sample whose solution became its cell's elite), whereas MT-ME and MONET revisit each task many times to refine its solution, so the total evaluation budget is identical across algorithms but the per-task refinement opportunity is not.
For PT-ME and MT-ME we use the default parameters from \cite{anneParametricTaskMAPElites2024,mouretQualitydiversitymultitaskoptimization2020}.
For MONET, we first conduct a hyperparameter sensitivity analysis.
We consider the following hyperparameters and ranges: $p_\text{ind} \in \{0,\, 0.3,\, 0.5,\, 0.7,\, 1\}$, $\text{strategy} \in \{\text{random},\, \text{best\_fitness}\}$, $\text{neighborhood} \in \{\text{random},\, \text{closest},\, \text{distance\_proportional}\}$, and $\text{neighbor\_percentage} \in \{1\%,\, 5\%,\, 10\%\}$. Here $p_\text{ind}$ is the per-step probability of performing individual learning (mutation) rather than social learning (SBX); $\text{strategy}$ is the rule used to pick the neighbor during social learning, either uniformly at random or deterministically the neighbor with the highest current fitness; $\text{neighborhood}$ determines how the neighbor set of each task is built, taking the $N$ closest tasks in $\mathcal{T}$, $N$ uniformly random tasks, or $N$ tasks drawn without replacement with probability proportional to task-space similarity; and $\text{neighbor\_percentage}$ sets this neighborhood size as a fraction $N = \lceil \text{neighbor\_percentage} \cdot |\mathcal{V}| \rceil$ of the total task set.
For each hyperparameter combination we perform up to 20 independent runs with different random seeds and report the mean fitness with standard deviation as well as the area under the learning curve (AUC), capturing both final performance and cumulative learning efficiency. The sensitivity sweep is concentrated on archery and arm (20 seeds per configuration; cartpole and hexapod are evaluated on a smaller subset of configurations because of simulator cost.
To quantify the importance of individual hyperparameters and their interactions, both per task and aggregated across all tasks, we fit a gradient-boosted tree surrogate model and compute SHapley Additive exPlanations (SHAP) values~\cite{lundberg2017unified,lundberg2020local2global}, following the surrogate-based hyperparameter importance methodology of~\cite{hutter2014efficient}.
Guided by this analysis, we then assign reasonable values to each hyperparameter, yielding a single configuration of MONET that we refer to as the \emph{default} configuration and compare it against PT-ME and MT-ME.
Additionally, we employ the Sequential Parameter Optimization Toolbox (SPOT)~\cite{bartzbeielstein2026optimizationspotoptim}, a surrogate-based optimizer, to automatically tune MONET's hyperparameters per task and include these task-specific configurations in the comparison.
To assess statistical significance of the performance differences between algorithms, we apply the Mann-Whitney U test~\cite{mann1947test} to the final fitness values and AUC values across all seeds and report the resulting $p$-values with Holm-Bonferroni correction~\cite{holm1979simple,dunn1961multiple} for multiple comparisons.
\begin{figure}[t]
\centering
% Row 1
\makebox[\textwidth]{\includegraphics[width=1\textwidth]{figs/all_tasks_hor.pdf}}
\caption{Overview of benchmark environments.}
\label{fig:benchmark-suite}
\end{figure}
Higher fitness is better in all four benchmarks, but the scale differs across them: archery and arm return values in $[0,1]$; cartpole reports the mean balanced-step count across 10 rollouts and therefore lies in $[0, t_{\max}]$ with $t_{\max}=1000$; hexapod reports the forward centre-of-mass displacement in metres over a 3\,s episode with no formal upper bound (in practice below $1.5\,\mathrm{m}$). Closed-form fitness expressions are defined in the Supplementary Information (SI)~\cite{monet_code}. All four benchmarks are run for $10^6$ evaluations and 20 seeds per run.
\paragraph{Archery.} A projectile-aiming task in which the archer compensates for target distance and wind (Figure~\ref{fig:benchmark-suite}a)~\cite{anneParametricTaskMAPElites2024}. Tasks (2D) vary the distance $D\in[5,40]$\,m and horizontal wind $W\in[-10,10]\,\mathrm{m/s^2}$; solutions (2D) are yaw and pitch in $[-\pi/12,\pi/12]$ at fixed launch speed $v_0=70\,\mathrm{m/s}$.
\paragraph{Arm.} A planar 10-degree-of-freedom (10-DoF) arm places its end-effector at a fixed target (Figure~\ref{fig:benchmark-suite}b)~\cite{mouretQualitydiversitymultitaskoptimization2020,anneParametricTaskMAPElites2024}. Tasks (2D) vary the link length $L$ and maximum joint angle $\alpha_{\max}$, both in $[0,1]$; solutions (10D) are the 10 joint angles.
\paragraph{Cartpole.} Cart-pole balancing under varying physical parameters (Figure~\ref{fig:benchmark-suite}c)~\cite{michie1968boxes,barto1983neuronlike}. Tasks (2D) vary pole mass $m_p\in[0.05,0.5]$\,kg and pole length $L\in[0.3,1.2]$\,m; solutions (16D) are a direct encoding of the weights of a small multi-layer perceptron controller (4 inputs, 2 hidden tanh units, 2 output logits, with bias units on the input and hidden layers).
\paragraph{Hexapod.} A simulated hexapod walks forward across 2\,000 random morphologies (Figure~\ref{fig:benchmark-suite}d)~\cite{MAP_Elites_NaturePaper,mouretQualitydiversitymultitaskoptimization2020}, simulated in PyBullet~\cite{coumans2021pybullet}. Tasks (12D) are 12 leg-segment length changes; solutions (36D) parameterise a gait controller.
\section{Results}\label{lab:results}
Across the four domains, MONET matches or exceeds the strongest MAP-Elites baseline on final mean fitness, with the largest gap on the hexapod task where the per-domain tuned configuration nearly doubles the final fitness of MT-ME (Figure~\ref{fig:algorithm_comparison}, Table~\ref{tab:results-summary}). We first walk through the per-domain comparison, then unpack the hyperparameter analysis that produced the shared default configuration.
\subsection{Algorithm Comparison}\label{sec:comparison}
We report two MONET variants: \emph{MONET$_{\mathrm{default}}$}, a single configuration shared across domains (defined in Section~\ref{sec:default-config}), and \emph{MONET$_{\mathrm{tuned}}$}, a per-domain tuning produced with SPOT \cite{bartzbeielstein2026optimizationspotoptim}. Both are compared against PT-ME and MT-ME with their published defaults. Each box in Figure~\ref{fig:algorithm_comparison} aggregates 20 seeds per (algorithm, domain) pair; brackets indicate pairwise comparisons where one algorithm is significantly better (Mann-Whitney U, Holm-Bonferroni correction, $p<0.05$).
\paragraph{Archery.} MT-ME, PT-ME, and MONET$_{\mathrm{tuned}}$ all saturate at $1.000$; MONET$_{\mathrm{default}}$ stops short at $0.996 \pm 0.000$ ($p<0.001$ vs.\ all three). Final fitness is thus tied at the top, and AUC is the discriminating signal. MONET$_{\mathrm{default}}$ reaches an AUC of $976.3 \pm 0.6$, significantly above MT-ME ($957.2 \pm 3.3$, $p<0.001$) and PT-ME ($924.1 \pm 1.1$, $p<0.001$). MONET$_{\mathrm{tuned}}$ improves AUC further to $979.2 \pm 1.9$ ($p<0.001$ vs.\ MONET$_{\mathrm{default}}$). MONET therefore converges to near-optimal performance substantially earlier than the baselines.
\paragraph{Arm.} MONET$_{\mathrm{default}}$ reaches $0.770 \pm 0.001$ on final fitness, just below MT-ME ($0.772 \pm 0.002$, $p<0.001$) and above PT-ME ($0.747 \pm 0.001$, $p<0.001$). On AUC it is the strongest of the four algorithms: $762.7 \pm 1.2$, ahead of MT-ME ($760.1 \pm 3.4$, $p=0.003$) and PT-ME ($734.0 \pm 2.9$, $p<0.001$). This reflects faster convergence in the early phase (Figure~\ref{fig:algorithm_comparison}b). MONET$_{\mathrm{tuned}}$ achieves the highest final fitness at $0.773 \pm 0.000$, significantly above MT-ME ($p<0.001$).
\paragraph{Cartpole.} MONET$_{\mathrm{default}}$ reaches $997.2 \pm 2.6$, statistically indistinguishable from MT-ME ($994.5 \pm 8.5$) and PT-ME ($992.5 \pm 8.9$). MT-ME retains a significant AUC lead ($978.4 \pm 11.4$ vs.\ $969.0 \pm 7.2$, $p<0.001$): it converges earlier while MONET terminates higher. Tuning nudges final fitness to $998.5 \pm 1.0$ (not significant vs.\ baselines) but significantly lowers AUC to $941.4 \pm 17.4$ ($p<0.001$ vs.\ MONET$_{\mathrm{default}}$).
\paragraph{Hexapod.} MONET$_{\mathrm{default}}$ ($0.690 \pm 0.101$) performs significantly better than MT-ME ($0.530 \pm 0.032$, $p<0.001$) and statistically indistinguishable from PT-ME ($0.727 \pm 0.078$, $p=0.15$). Tuning improves performance further to $0.956 \pm 0.082$, significantly above both baselines ($p<0.001$ vs.\ MT-ME; $p<0.001$ vs.\ PT-ME).
%When the parameters of MONET are The hexapod locomotion task shows the largest separation. %MONET$_{\mathrm{tuned}}$ reaches $0.962 \pm 0.142$, against $0.727 \pm 0.078$ for PT-ME and $0.530 \pm 0.032$ for MT-ME, with the same ordering on AUC. We attribute the gap to the diversity of the 2\,000 morphologies: MONET's task-graph routing transfers solutions between similar morphologies, whereas the flat archives of PT-ME and MT-ME share information indiscriminately. \textcolor{red}{MONET tuned results were not provided.}
Previous work reported that PT-ME outperformed MT-ME~\cite{anneParametricTaskMAPElites2024}, whereas in our experiments, MT-ME achieved better performance than PT-ME. This discrepancy arises because PT-ME samples tasks continuously from the task space, which can result in selecting tasks that are easier to solve. To ensure a fair comparison, we fixed the task set to the final tasks discovered by PT-ME and evaluated both methods on this same set of tasks.
\begin{figure}[!t]
\centering
\includegraphics[width=\textwidth]{figs/algorithm_comparison.pdf}
\caption{Algorithm comparison across the four benchmark domains.}
\label{fig:algorithm_comparison}
\end{figure}
\subsection{Hyperparameter Sensitivity and Default Configuration}\label{sec:default-config}\label{sec:sensitivity}
The most important finding from the SHAP-based sensitivity analysis is that the probability of individual learning $p_\text{ind}$ accounts for most of the variation in performance, with a mean absolute SHAP value $\overline{|\text{SHAP}|}=0.49$, followed by $\text{neighbor\_percentage}$ ($0.16$), $\text{strategy}$ ($0.13$), and $\text{neighborhood}$ ($0.09$); see Figure~\ref{fig:shap-global}. Low values of $p_\text{ind}$ (favoring social learning over individual mutation) consistently improve both mean fitness and AUC across all domains, supporting the central design intuition behind MONET that information transfer between similar tasks is the dominant lever. The remaining hyperparameters show weaker and more domain-dependent effects (see SI for detailed SHAP analysis per domain~\cite{monet_code}).
The default parameters of MONET were defined based on the SHAP analysis as: strategy $=$ best\_fitness, neighborhood $=$ closest, $p_\text{ind}=0.3$, and neighbor\_percentage $=5\%$ (obtained by ranking configurations on the z-scored mean fitness across all domains, see SI~\cite{monet_code}). This configuration picks the fittest neighbor for crossover, draws neighbors from the $k$ nearest tasks in task space, and mixes mostly social learning ($70\%$) with occasional individual refinement ($30\%$). The SPOT-tuned per-domain configurations differ markedly from the default and from each other: archery uses a large neighborhood ($39\%$) with moderate $p_\text{ind}=0.22$; arm uses a very small neighborhood ($0.1\%$) with high $p_\text{ind}=0.54$ (favouring individual refinement on this well-conditioned task); cartpole uses a small neighborhood ($1\%$) with a balanced $p_\text{ind}=0.5$; and hexapod uses a small neighborhood ($1\%$) with pure social learning ($p_\text{ind}=0$). This spread confirms that the optimal individual vs.\ social learning balance is task-dependent. The default configuration is competitive with the baselines and achieves the best AUC on three of four domains (archery, arm, hexapod); it is outperformed only on archery final fitness, where PT-ME and MT-ME saturate at $1.000$, and on cartpole AUC, where MT-ME converges faster (Fig.~\ref{fig:algorithm_comparison}, Table~\ref{tab:results-summary})
\begin{figure}[t]
\centering
\includegraphics[width=0.7\linewidth]{figs/shap_global_importance.pdf}
\caption{Global hyperparameter importance for MONET: mean absolute SHAP value on a gradient-boosted tree surrogate.}
\label{fig:shap-global}
\end{figure}
\begin{table}[!ht]
\centering
\caption{Final mean fitness and AUC ($\times 10^3$ on the $[0,1]$ domains) per domain. Mean $\pm$ std over 20 random seeds per (algorithm, domain) pair.}
\label{tab:results-summary}
\small
\setlength{\tabcolsep}{3pt}
\begin{tabular}{@{}lcccc@{}}
\toprule
& Archery & Arm & Cartpole & Hexapod \\
\midrule
\multicolumn{5}{@{}l}{\textit{(a) Final mean fitness}} \\[2pt]
PT-ME & $1.000 \pm 0.000$ & $0.747 \pm 0.001$ & $992.5 \pm 8.9$ & $0.727 \pm 0.078$ \\
MT-ME & $1.000 \pm 0.000$ & $0.772 \pm 0.002$ & $994.5 \pm 8.5$ & $0.530 \pm 0.032$ \\
MONET$_{\mathrm{default}}$ & $0.996 \pm 0.000$ & $0.770 \pm 0.001$ & $997.2 \pm 2.6$ & $0.690 \pm 0.101$ \\
MONET$_{\mathrm{tuned}}$ & $1.000 \pm 0.000$ & $\mathbf{0.773 \pm 0.000}$ & $998.5 \pm 1.0$ & $\mathbf{0.957 \pm 0.083}$ \\
\midrule
\multicolumn{5}{@{}l}{\textit{(b) AUC ($\times 10^3$ on $[0,1]$ domains)}} \\[2pt]
PT-ME & $924.1 \pm 1.1$ & $734.0 \pm 2.9$ & $913.5 \pm 70.2$ & $349.9 \pm 27.7$ \\
MT-ME & $957.2 \pm 3.3$ & $760.1 \pm 3.4$ & $\mathbf{978.4 \pm 11.4}$ & $450.2 \pm 29.3$ \\
MONET$_{\mathrm{default}}$ & $976.3 \pm 0.6$ & $\mathbf{762.7 \pm 1.2}$ & $969.0 \pm 7.2$ & $557.0 \pm 67.3$ \\
MONET$_{\mathrm{tuned}}$ & $\mathbf{979.2 \pm 1.9}$ & $757.3 \pm 0.2$ & $941.4 \pm 17.4$ & $\mathbf{716.0 \pm 87.8}$ \\
\bottomrule
\end{tabular}
\end{table}
\section{Conclusion}\label{lab:discussion}
In this work, we propose MONET to find high-quality solutions for a large set of related tasks simultaneously and in parallel. We propose structuring the task space as a network to establish topological relations and neighborhoods, which aims to exploit task similarities for knowledge transfer. As shown by the results in four domains, this approach has an advantage over flat archive representation, where these topological relations are ignored.
We introduced two operators, namely individual and social learning, that aim to improve tasks in isolation or exploit knowledge transfer across neighboring related tasks.
The extensive sensitivity analysis of the MONET algorithm and its parameters showed that the probability of individual learning $p_\text{ind}$ influences the performance the most, with high values leading to lower performance. This shows that
social learning is a significant contributor to performance by exploiting knowledge transfer across tasks. However, this analysis was on a pre-selected set of $p_\text{ind}$ parameters. Thus, using a more balanced ratio between individual and social learning parameters can improve performance, particularly in task spaces
where similarities between tasks are relatively low. In fact, SPOT-based per-domain tuning selected low $p_\text{ind}$ on hexapod $(0)$ and archery $(0.54)$. On arm $(0.5)$ and cartpole $0.5$, a more balanced mix was preferred.
We establish connectivity between tasks on the basis of the task parameter similarities. This relies on the idea that tasks sharing similar parameters are themselves similar, making them more likely to benefit from knowledge transfer and to converge toward similar solutions.
However, this assumption may not hold in many domains. In the hexapod domain, for example, small changes to front-right and back-left leg lengths yield nearby task vectors but can demand qualitatively different solutions to perform the control tasks. Future work should therefore explore alternative connectivity definitions and assess their effect on knowledge transfer.
Currently, the MONET algorithm has four hyperparameters. Another interesting direction is online self-adaptive parameter control (e.g., adapting the neighborhood size as in MT-ME's task-competition and replacement rule~\cite{mouretQualitydiversitymultitaskoptimization2020}, or dynamically switching between individual and social learning strategies~\cite{yaman2022meta}). Although this can introduce complexity, it may provide more generalized performance across different domains and task spaces.
Another promising direction is to extend the types of individual and social
learning strategies that can improve performance. Furthermore, some of these
strategies can converge to more general solutions that can lead to satisfactory performance for a wide range of tasks in the task space ~\cite{triebold2024evolving}.
%Several limitations qualify these results. First, the task graph is a fixed $k$-NN structure under Euclidean distance in $\mathcal{T}$, which is appropriate for our four task spaces but may not generalize to domains where task similarity is non-metric or learned from data. Second, some (algorithm, domain) cells are under-seeded because of simulator cost (e.g.\ MONET$_{\mathrm{tuned}}$ on cartpole, $n=5$); the effect sizes relative to MT-ME and PT-ME remain significant under Mann-Whitney U, but additional seeds would tighten those estimates. Third, the default configuration has not yet been run on cartpole and hexapod, so the case for a single shared configuration rests on the archery and arm results plus the SPOT comparison.
%Natural extensions follow from each of these. Replacing the static $k$-NN graph with an adaptive construction — for instance, edges learned from solution-space similarity rather than task-space distance, or rewired during optimization based on transfer success — would let MONET exploit task structure that is not visible in $\mathcal{T}$. Combining the graph with a behavioral descriptor (in the spirit of MTMB-ME) would allow MONET to maintain both task-specific elites and within-task diversity. Finally, scaling beyond the $\mathcal{O}(10^3)$ task regime tested here is a clear next step: the $\mathcal{O}(|\mathcal{V}|\,k)$ memory of the graph already permits much larger $|\mathcal{V}|$ than the discretized archives it replaces.
\section{Node Coverage Guarantees Under Random Sampling}
If we draw at each evaluation step a random node and perform an action, we may ask how many evaluations are required so that we visit each of the \(\nu\) nodes at least once with high certainty.
This problem can be framed as a \emph{coupon collector's problem}~\cite{feller1968introduction}. Writing $M$ for the number of draws, the expected number of steps needed to visit all \(\nu\) nodes at least once is
\[
\mathbb{E}[M]
= \nu H_\nu
= \nu \left( \ln \nu + \gamma + O\!\left(\frac{1}{\nu}\right) \right),
\]
where \(H_\nu\) is the \(\nu\)-th harmonic number and \(\gamma \approx 0.57721\) is the Euler-Mascheroni constant.
\bigskip
Beyond the expectation, one may also ask for a high-probability guarantee.
Using the asymptotic limit for the coupon collector's completion time,
the probability that all nodes have been visited by draw
\[
M = \nu(\ln \nu + c)
\]
converges to
\[
\mathbb{P}\bigl(M \le \nu(\ln \nu + c)\bigr) \approx e^{-e^{-c}}.
\]
To achieve at least \(99\%\) probability of having visited all nodes, we solve for $c$
\[
\begin{aligned}
e^{-e^{-c}} &= p \\[8pt]
\ln\!\left(e^{-e^{-c}}\right) &= \ln(p) \\[8pt]
-e^{-c} &= \ln(p) \\[8pt]
\text{For } p = 0.99:\quad
c &= -\ln\!\bigl(-\ln(0.99)\bigr) \approx 4.60015 \approx \ln(100).
\end{aligned}
\]
Since each node is selected independently with probability $\tfrac{1}{\nu}$ at every draw,
the visit count of any particular node over $M$ draws follows a Binomial distribution,
\[
X_i \sim \mathrm{Binomial}\!\left(M, \tfrac{1}{\nu}\right),
\]
from which the expected number of visits per node is
\[
\mathbb{E}[X_i] = \frac{M}{\nu}.
\]
We evaluate four environments with differing numbers of distinct tasks and a fixed amount of draws (number of evaluations) $M=1{,}000{,}000$:
\begin{table}[h]
\centering
\caption{Coupon collector bounds for the four benchmark environments with $M=1{,}000{,}000$ total evaluations.}
\label{tab:coupon-collector}
\begin{tabular}{@{}lrrrr@{}}
\toprule
Environment & $\nu$ & $\mathbb{E}[M]$ & $M_{0.99}$ & $\mathbb{E}[X_i]$ \\
\midrule
Archery / Arm / Cartpole & 5\,000 & 45\,472 & 65\,612 & 200 \\
Hexapod & 2\,000 & 16\,356 & 24\,412 & 500 \\
\bottomrule
\end{tabular}
\end{table}
\section{MONET Algorithm}
Algorithm~\ref{alg:individual} and Algorithm~\ref{alg:social} detail the two learning subroutines invoked in Algorithm~\ref{alg:monet}.
\paragraph{Individual Learning.}
When individual learning is selected, the current elite at node $\boldsymbol{\tau}_i$ is mutated independently of any neighbor. If no solution exists at the node, a random solution is drawn from $\mathcal{U}(p_{\min}, p_{\max})$ and evaluated. Otherwise, Gaussian mutation with standard deviation $\sigma_m$ scaled by the parameter range is applied. The offspring replaces the current elite only if its fitness is at least as high.
\begin{algorithm}[H]
\caption{Individual Learning}\label{alg:individual}
\begin{algorithmic}[1]
\Require Task $\boldsymbol{\tau}$, archive $\mathcal{A}$, bounds $p_{\min}, p_{\max}$, mutation std $\sigma_m$
\Ensure Updated $\mathcal{A}[\boldsymbol{\tau}]$
\If{$\mathcal{A}[\boldsymbol{\tau}] = \emptyset$} \Comment{initialize empty node}
\State $\boldsymbol{\theta} \sim \mathcal{U}(p_{\min}, p_{\max})$ \Comment{random solution}
\State $\mathcal{A}[\boldsymbol{\tau}] \gets (\boldsymbol{\theta},\, f(\boldsymbol{\theta}, \boldsymbol{\tau}))$
\State \Return
\EndIf
\State $\boldsymbol{\theta} \gets \mathcal{A}[\boldsymbol{\tau}].\text{solution}$ \Comment{current elite}
\State $\boldsymbol{\epsilon} \sim \mathcal{N}(\mathbf{0},\, \sigma_m (p_{\max} - p_{\min}))$ \Comment{Gaussian perturbation}
\State $\mathbf{y} \gets \text{clip}(\boldsymbol{\theta} + \boldsymbol{\epsilon},\; p_{\min},\; p_{\max})$ \Comment{mutant offspring}
\State $f_y \gets f(\mathbf{y}, \boldsymbol{\tau})$
\If{$f_y \geq \mathcal{A}[\boldsymbol{\tau}].\text{fitness}$} \Comment{elitist replacement}
\State $\mathcal{A}[\boldsymbol{\tau}] \gets (\mathbf{y}, f_y)$
\EndIf
\end{algorithmic}
\end{algorithm}
\paragraph{Social Learning.}
When social learning is selected, the current elite is recombined with a solution drawn from a neighboring node. If the current node is empty, the neighbor's solution is copied as an initial seed (or a random solution is used when no neighbor has a solution). Recombination is performed via Simulated Binary Crossover (SBX)~\cite{SBX} with distribution index $\eta_c=10$~\cite{deb2001multi}. As with individual learning, the offspring replaces the elite only upon improvement.
\begin{algorithm}[H]
\caption{Social Learning}\label{alg:social}
\begin{algorithmic}[1]
\Require Task $\boldsymbol{\tau}$, neighbor task $\boldsymbol{\tau}_j$, archive $\mathcal{A}$, bounds $p_{\min}, p_{\max}$
\Ensure Updated $\mathcal{A}[\boldsymbol{\tau}]$
\If{$\mathcal{A}[\boldsymbol{\tau}] = \emptyset$} \Comment{initialize empty node}
\If{$\mathcal{A}[\boldsymbol{\tau}_j] \neq \emptyset$}
\State $\boldsymbol{\theta} \gets \mathcal{A}[\boldsymbol{\tau}_j].\text{solution}$ \Comment{copy from neighbor}
\Else
\State $\boldsymbol{\theta} \sim \mathcal{U}(p_{\min}, p_{\max})$ \Comment{random fallback}
\EndIf
\State $\mathcal{A}[\boldsymbol{\tau}] \gets (\boldsymbol{\theta},\, f(\boldsymbol{\theta}, \boldsymbol{\tau}))$
\State \Return
\EndIf
\If{$\mathcal{A}[\boldsymbol{\tau}_j] = \emptyset$}
\State \Return \Comment{no valid neighbor solution}
\EndIf
\State $\mathbf{y} \gets \text{SBX}\bigl(\mathcal{A}[\boldsymbol{\tau}].\text{solution},\; \mathcal{A}[\boldsymbol{\tau}_j].\text{solution},\; \eta_c\!=\!10\bigr)$ \Comment{crossover}
\State $\mathbf{y} \gets \text{clip}(\mathbf{y},\, p_{\min},\, p_{\max})$
\State $f_y \gets f(\mathbf{y}, \boldsymbol{\tau})$
\If{$f_y \geq \mathcal{A}[\boldsymbol{\tau}].\text{fitness}$} \Comment{elitist replacement}
\State $\mathcal{A}[\boldsymbol{\tau}] \gets (\mathbf{y}, f_y)$
\EndIf
\end{algorithmic}
\end{algorithm}
\section{Closed-Form Task Fitness Definitions}\label{app:tasks}
This appendix gives the explicit fitness expressions for the four benchmarks.
\subsection{Archery}
With initial velocity vector
\[
\mathbf{v} = v_0
\begin{bmatrix}
-\sin(\text{yaw}) \\
\cos(\text{yaw})\cos(\text{pitch}) \\
\cos(\text{yaw})\sin(\text{pitch})
\end{bmatrix},
\quad v_0 = 70\,\mathrm{m/s},
\]
the target-plane hit time is $t_{\text{hit}} = D/v_y$ whenever $v_y > 0$; if $v_y \le 0$ the shot misses and $f = 0$. Taking $g = 9.8\,\mathrm{m/s^2}$ and wind $W\in[-10,10]\,\mathrm{m/s^2}$ along $x$, the miss coordinates on the target plane ($y = D$) are
\[
m_x = \tfrac{1}{2}\,W\,t_{\text{hit}}^2 + v_x\,t_{\text{hit}},
\qquad
m_z = -\tfrac{1}{2}\,g\,t_{\text{hit}}^2 + v_z\,t_{\text{hit}},
\qquad
r = \sqrt{m_x^2 + m_z^2}.
\]
Fitness is computed on a 10-ring target of ring width $0.061\,\mathrm{m}$:
\[
f = \frac{\max\!\bigl(0,\; 10 - \lfloor r/0.061 \rfloor\bigr)}{10} \in \{0, 0.1, \ldots, 1.0\}.
\]
\subsection{Arm}
With end-effector position $\mathbf{p}_{\text{ee}}\in\mathbb{R}^2$ and a fixed target $\mathbf{p}_{\text{tar}} = (0.5, 0.5)$,
\[
f = \exp\!\left(-\lVert \mathbf{p}_{\text{ee}} - \mathbf{p}_{\text{tar}} \rVert_2\right) \in (0,1].
\]
The joint commands are $\mathbf{q} = (\boldsymbol{\theta}-\tfrac{1}{2})\cdot 2\pi\,\alpha_{\max}/10$ where $\boldsymbol{\theta}\in[0,1]^{10}$ is the solution vector and $\alpha_{\max}$ is the task-dependent maximum joint-angle fraction; each of the $10$ links has length $L/10$ with $L$ the task-dependent total arm length. $\mathbf{p}_{\text{ee}}$ is the end-effector position returned by forward kinematics.
\subsection{Cartpole}
The task uses the \texttt{CartPole-v1} dynamics from OpenAI Gym with the pole mass $m_p$ and pole length $L$ replaced by the per-task values; the default cart mass $m_c$ is retained. With pole angle $\phi$, cart position $x_c$, control force $F\in\{-10, +10\}\,\mathrm{N}$, and gravity $g=9.8\,\mathrm{m/s^2}$,
\[
\ddot{\phi} = \frac{g \sin \phi + \cos \phi \left(-F - m_p L \dot{\phi}^2 \sin \phi\right)/(m_c + m_p)}{L \left(\tfrac{4}{3} - \tfrac{m_p \cos^2 \phi}{m_c + m_p}\right)},
\qquad
\ddot{x}_c = \frac{F + m_p L \left(\dot{\phi}^2 \sin \phi - \ddot{\phi} \cos \phi\right)}{m_c + m_p}.
\]
A solution is evaluated over $n_{\text{rollouts}} = 10$ rollouts with distinct Gym seeds. Each rollout runs until the pole leaves the balanced region ($|\phi| \ge 12^\circ$ or $|x_c| \ge 2.4\,\mathrm{m}$) or until the step cap $t_{\max}=1000$ is reached. Writing $t_{\text{balanced}}^{(k)}$ for the number of surviving steps on rollout $k$, the reported fitness is the arithmetic mean
\[
f = \frac{1}{n_{\text{rollouts}}}\sum_{k=1}^{n_{\text{rollouts}}} \min\!\bigl(t_{\max},\, t_{\text{balanced}}^{(k)}\bigr) \in [0, t_{\max}].
\]
\subsection{Hexapod}
With gait parameters $\boldsymbol{\theta}\in[0,1]^{36}$ and morphology $\boldsymbol{\tau}\in\mathcal{T}$ instantiated through a per-task URDF, a PyBullet simulation is run for up to $3\,\mathrm{s}$ of wall-clock simulator time. The episode terminates early if any body-axis angle (roll, pitch, yaw) exceeds $\pi/8$ or if the lateral deviation satisfies $|y|>0.5\,\mathrm{m}$. Writing $p_x(t)$ for the forward position of the center of mass and $T\le 3\,\mathrm{s}$ for the time of termination or episode end,
\[
f(\boldsymbol{\theta},\boldsymbol{\tau}) = p_x(T),
\]
reported in metres. The controller returns $f=0$ if any entry of $\boldsymbol{\theta}$ lies outside $[0,1]$.
\section{Detailed Sensitivity Analysis}\label{app:sensitivity}
This appendix provides per-domain SHAP beeswarm plots and the SPOT-tuned hyperparameter configurations that complement the global sensitivity analysis.
\subsection{Per-Domain SHAP Analysis}
Figures~\ref{fig:shap-archery} and~\ref{fig:shap-arm} show the SHAP beeswarm plots for the archery and arm domains, respectively. Each point represents one MONET run; its horizontal position indicates the SHAP value (contribution to the predicted metric), and its color encodes the feature value (red = high, blue = low).
In both domains, $p_\text{ind}$ exhibits the strongest effect: high values of $p_\text{ind}$ (more individual learning) consistently decrease predicted fitness, corroborating the global finding that social learning is the primary driver of performance. The remaining parameters show smaller, more task-specific effects.
\begin{figure}[H]
\centering
\includegraphics[width=1\linewidth]{figs/monet_hyperparam_importance_archery.pdf}
\caption{SHAP beeswarm plot for the archery domain. Each row is a hyperparameter; each dot is a MONET run. Color indicates the feature value (red = high, blue = low); horizontal position indicates the SHAP value.}
\label{fig:shap-archery}
\end{figure}
\begin{figure}[H]
\centering
\includegraphics[width=1\linewidth]{figs/monet_hyperparam_importance_arm.pdf}
\caption{SHAP beeswarm plot for the arm domain. Panels as in Figure~\ref{fig:shap-archery}.}
\label{fig:shap-arm}
\end{figure}
\begin{figure}[H]
\centering
\includegraphics[width=1\linewidth]{figs/monet_hyperparam_importance_cartpole.pdf}
\caption{SHAP beeswarm plot for the cartpole domain. Panels as in Figure~\ref{fig:shap-archery}.}
\label{fig:shap-cartpole}
\end{figure}
\begin{figure}[H]
\centering
\includegraphics[width=1\linewidth]{figs/monet_hyperparam_importance_hexapod.pdf}
\caption{SHAP beeswarm plot for the hexapod domain. Panels as in Figure~\ref{fig:shap-archery}.}
\label{fig:shap-hexapod}
\end{figure}
\subsection{SPOT-Tuned Configurations}
Table~\ref{tab:spot-configs} lists the per-domain configurations found by the SPOT hyperparameter tuner.
\begin{table}[h]
\centering
\caption{SPOT-tuned MONET configurations per domain.}
\label{tab:spot-configs}
\small
\begin{tabular}{@{}lllcc@{}}
\toprule
\textbf{Domain} & \textbf{strategy} & \textbf{neighborhood} & $p_\text{ind}$ & \textbf{neighbor\_pct.} \\
\midrule
Archery & best\_fitness & closest & 0.0 & 2\% \\
Arm & random & closest & 0.3 & 1\% \\
Cartpole & best\_fitness & random & 0.5 & 1\% \\
Hexapod & best\_fitness & random & 0.0 & 1\% \\
\bottomrule
\end{tabular}
\end{table}
\section{MONET Hyperparameter Descriptions}\label{app:hyperparameters}
Table~\ref{tab:monet-hyperparams} summarizes the four hyperparameters varied in the sensitivity analysis. All other algorithmic parameters (mutation operator, crossover operator, initialization strategy) are held fixed throughout. See Algorithm~\ref{alg:monet} for the points at which each parameter enters the loop.
\begin{table}[h]
\centering
\caption{MONET hyperparameters and their roles.}
\label{tab:monet-hyperparams}
\renewcommand{\arraystretch}{1.3}
\small
\begin{tabular}{@{}p{2.4cm}p{2.2cm}p{7.4cm}@{}}
\toprule
\textbf{Parameter} & \textbf{Values} & \textbf{Description} \\
\midrule
$p_\text{ind}$ &
0, 0.3, 0.5, 0.7, 1 &
Probability that a given evaluation step performs \emph{individual learning} (mutation of the focal task's own elite) rather than \emph{social learning} (crossover with a neighbor's elite). At $p_\text{ind}=0$ only social learning occurs; at $p_\text{ind}=1$ only individual learning occurs. \\
\addlinespace
strategy &
random, best\_fitness &
Neighbor selection strategy during social learning. \texttt{random}: uniform random selection among all neighbors. \texttt{best\_fitness}: deterministically select the neighbor with the highest current fitness. \\
\addlinespace
neighborhood &
random, closest, distance\_prop. &
Method for constructing the neighbor set of each task.
\texttt{closest}: the $N$ nearest tasks by Euclidean distance in task space.
\texttt{random}: $N$ tasks sampled uniformly at random (sorted by similarity after sampling).
\texttt{distance\_proportional}: $N$ tasks sampled \emph{without replacement} with probability proportional to task-space similarity $s(\boldsymbol{\tau}_i, \boldsymbol{\tau}_j) = 1/(1 + \lVert \boldsymbol{\tau}_i - \boldsymbol{\tau}_j \rVert)$; this softly biases the neighborhood toward nearby tasks while retaining a chance of including distant ones. \\
\addlinespace
neighbor\_pct. &
1\%, 5\%, 10\% &
Fraction of the total task set used as the neighborhood size~$N = \lceil \texttt{neighbor\_percentage} \cdot |\mathcal{V}| \rceil$. For instance, with 5\,000 tasks and 5\%, each task has $N=250$ neighbors. Controls graph sparsity: lower values yield sparser graphs with more localized knowledge transfer. \\
\bottomrule
\end{tabular}
\end{table}
\section{Hyperparameter Configuration Ranking}\label{app:ranking}
Table~\ref{tab:top10} lists the ten highest-ranked MONET hyperparameter configurations (across the 2--4 domains on which they were run) by combined z-score, computed as the mean of the within-domain z-scores of final mean fitness and AUC. Table~\ref{tab:bottom5} shows the five worst-ranked configurations for comparison. Only configurations evaluated on at least two domains are included in the ranking. The counts $n_{\text{Arm}}, n_{\text{Archery}}, n_{\text{Cartpole}}, n_{\text{Hexapod}}$ report the number of deduplicated seed runs per domain.
\begin{table}[h]
\centering
\caption{Top 10 MONET configurations by combined z-score ($n_{\text{domains}} \ge 2$).}
\label{tab:top10}
\footnotesize
\setlength{\tabcolsep}{3pt}
\begin{tabular}{@{}llrrrrrrrrr@{}}
\toprule
strategy & neighborhood & nbr.\% & $p_\text{ind}$ & $n_\text{Arm}$ & $n_\text{Arc}$ & $n_\text{Cart}$ & $n_\text{Hex}$ & $z_{\text{FF}}$ & $z_{\text{AUC}}$ & combined \\
\midrule
best\_fitness & closest & 1.5 & 0.30 & 20 & 20 & 0 & 0 & $+0.58$ & $+1.01$ & $\mathbf{+0.80}$ \\
best\_fitness & closest & 1.0 & 0.30 & 22 & 22 & 0 & 5 & $+0.60$ & $+0.96$ & $+0.78$ \\
best\_fitness & closest & 2.0 & 0.30 & 20 & 20 & 0 & 0 & $+0.55$ & $+0.99$ & $+0.77$ \\
best\_fitness & closest & 2.0 & 0.50 & 20 & 39 & 5 & 0 & $+0.62$ & $+0.90$ & $+0.76$ \\
best\_fitness & closest & 1.5 & 0.70 & 5 & 20 & 0 & 0 & $+0.62$ & $+0.81$ & $+0.72$ \\
best\_fitness & closest & 2.0 & 0.70 & 5 & 20 & 0 & 0 & $+0.62$ & $+0.82$ & $+0.72$ \\
random & closest & 2.0 & 0.00 & 20 & 19 & 4 & 0 & $+0.66$ & $+0.75$ & $+0.71$ \\
random & closest & 1.5 & 0.00 & 20 & 19 & 5 & 0 & $+0.66$ & $+0.75$ & $+0.70$ \\
best\_fitness & closest & 1.0 & 0.50 & 40 & 41 & 5 & 5 & $+0.56$ & $+0.82$ & $+0.69$ \\
random & closest & 1.0 & 0.00 & 40 & 40 & 5 & 3 & $+0.64$ & $+0.73$ & $+0.68$ \\
\bottomrule
\end{tabular}
\end{table}
\begin{table}[h]
\centering
\caption{Bottom 5 MONET configurations by combined z-score ($n_{\text{domains}} \ge 2$).}
\label{tab:bottom5}
\footnotesize
\setlength{\tabcolsep}{3pt}
\begin{tabular}{@{}llrrrrrrrrr@{}}
\toprule
strategy & neighborhood & nbr.\% & $p_\text{ind}$ & $n_\text{Arm}$ & $n_\text{Arc}$ & $n_\text{Cart}$ & $n_\text{Hex}$ & $z_{\text{FF}}$ & $z_{\text{AUC}}$ & combined \\
\midrule
best\_fitness & distance\_prop.\ & 5.0 & 1.00 & 20 & 20 & 0 & 2 & $-1.22$ & $-1.86$ & $-1.54$ \\
best\_fitness & distance\_prop.\ & 10.0 & 0.00 & 20 & 20 & 0 & 16 & $-2.51$ & $-0.65$ & $-1.58$ \\
random & closest & 34.3 & 0.98 & 2 & 1 & 1 & 0 & $-1.19$ & $-1.97$ & $-1.58$ \\
best\_fitness & distance\_prop.\ & 5.0 & 0.00 & 20 & 20 & 0 & 3 & $-3.00$ & $-0.93$ & $-1.96$ \\
best\_fitness & distance\_prop.\ & 41.5 & 0.99 & 2 & 2 & 2 & 0 & $-1.97$ & $-2.23$ & $\mathbf{-2.10}$ \\
\bottomrule
\end{tabular}
\end{table}
Two patterns are clear from Table~\ref{tab:top10}: (i)~the \texttt{closest} neighborhood dominates the top of the ranking, and (ii)~small neighborhood sizes ($\le 2\%$ of the task set) are preferred. Poorly-performing configurations (Table~\ref{tab:bottom5}) combine the \texttt{distance\_proportional} neighborhood with either very high $p_\text{ind}$ (near pure individual learning) or very large neighborhoods, both of which disrupt local knowledge transfer.
\section{Code Availability}\label{app:code}
Table~\ref{tab:code-availability} lists the reference papers and public code repositories for MONET and the three baselines.
\begin{table}[h]
\centering
\caption{Reference papers and public code repositories for MONET and the baseline algorithms.}
\label{tab:code-availability}
\small
\setlength{\tabcolsep}{4pt}
\begin{tabular}{@{}lll@{}}
\toprule
\textbf{Algorithm} & \textbf{Paper} & \textbf{Code} \\
\midrule
MONET (ours) & This paper & \cite{monet_code} \\
MAP-Elites & \cite{DBLP:journals/corr/MouretC15,MAP_Elites_NaturePaper} & \cite{pymap_elites_code} \\
MT-ME & \cite{mouretQualitydiversitymultitaskoptimization2020} & \cite{pymap_elites_code} \\
PT-ME & \cite{anneParametricTaskMAPElites2024} & \cite{pt_me_code} \\
\bottomrule
\end{tabular}
\end{table}