Reward-Rate Congestion Games and Replicator–Dinkelbach Dynamics

Abstract

Reward rate is a key performance criterion in cyber-physical and robotic systems where time, workload, and coordination costs are limiting resources. We introduce reward-rate congestion games, where agents seek to maximize reward per unit execution time. The direct reward-rate game is generally not an exact potential game. We develop a Dinkelbach-based framework in which, for every fixed Dinkelbach parameter, the transformed game is an exact potential game. This yields a potential-level Dinkelbach iteration that terminates finitely at the optimal potential reward rate when the inner potential maximization problem is solved globally. We also provide a sufficient condition under which an equilibrium of the transformed game is an equilibrium of the original reward-rate game. To optimize aggregate performance, we introduce marginal externality corrections that make the corrected potential coincide with the Dinkelbach-transformed social reward-rate objective, thereby enabling optimization of the social reward rate. Finally, we develop a continuous-time replicator–Dinkelbach dynamics for reward-rate population games coupling fast replicator dynamics with a slow reward-rate update. We establish convergence of the fixed-parameter replicator dynamics, global asymptotic and local exponential stability of the reduced Dinkelbach dynamics, and local exponential stability of the coupled system for sufficiently slow Dinkelbach updates. The framework is illustrated on a continuous task-allocation problem.

I Introduction

Congestion games provide a fundamental model for multi-agent decision-making over shared resources. Since their introduction by Rosenthal [1], congestion games have been used to model routing, electrical grids, and network resource allocation problems [2, 3, 4]. A central feature of congestion games is their connection to potential games: unilateral payoff variations can be represented by variations of a common potential function [5]. This potential structure guarantees the existence of pure Nash equilibria and provides a natural basis for equilibrium computation and learning.

In many cyber-physical and robotic systems, however, performance is more naturally measured by reward per unit execution time than by total reward. Examples include search-and-rescue, environmental monitoring, warehouse automation, and inspection, where performance is measured by rates of detection, coverage, or task completion. Similar considerations arise for human–robot teams, where task decisions must account for robot execution time, human workload, and coordination costs. Thus, high total reward alone is insufficient when achieving it requires excessive time or human effort.

Reward-rate objectives introduce a structural difficulty. Standard congestion games are compatible with exact-potential methods because additive resource payoffs aggregate into a Rosenthal-type potential. In contrast, reward-rate payoffs consider the ratio of reward to execution time. Such ratio payoffs do not generally preserve the standard exact-potential structure of congestion games. Consequently, decentralized learning and equilibrium computation methods for potential games cannot be applied directly to reward-rate congestion games.

This loss of potential structure is particularly important for learning and equilibrium computation, since the common potential function provides a global objective for analyzing many decentralized adjustment dynamics [6, 7, 8, 9, 10, 11, 12]. It therefore motivates a transformation that recovers potential structure while retaining the reward–time tradeoff. To address this challenge, we use the Dinkelbach transformation [13]. For each fixed value of the Dinkelbach parameter, the transformation replaces the reward-rate ratio by a reward-minus-time objective that is additive across resources, thereby recovering an exact-potential game. This structure enables a potential-level fractional optimization framework and, with appropriate externality corrections, optimization of the social reward rate.

The main contributions of this paper are as follows. First, we introduce reward-rate congestion games, where agents seek to maximize reward per unit execution time, and show that the Dinkelbach-transformed game is an exact potential game for every fixed Dinkelbach parameter. This yields a potential-level Dinkelbach iteration that terminates finitely at the optimal potential reward rate whenever the inner problem is solved globally. Second, we introduce marginal externality corrections that make the potential of the corrected transformed game coincide with the Dinkelbach-transformed social reward-rate objective. Thus, optimizing the corrected potential optimizes the social reward rate. Third, we develop a continuous-time replicator–Dinkelbach dynamics for reward-rate population games. We establish convergence of the fixed-parameter replicator dynamics, global asymptotic and local exponential stability of the reduced Dinkelbach dynamics, and local exponential stability of the coupled system for sufficiently slow Dinkelbach updates. The framework is illustrated on a continuous population task-allocation problem.

II Reward-Rate Congestion Games

Consider a finite set of agents $\mathcal{N}=\{1,\dots,N\}$ and a finite set of resources $\mathcal{E}$. Each agent $i\in\mathcal{N}$ selects an action $a_{i}\in\mathcal{A}_{i}$, where $\mathcal{A}_{i}\subseteq 2^{\mathcal{E}}$. Let $a=(a_{1},\dots,a_{N})\in\mathcal{A}:=\prod_{i=1}^{N}\mathcal{A}_{i}$ denote an action profile. For each resource $e\in\mathcal{E}$, define the corresponding congestion level by $n_{e}(a):=\left|\{i\in\mathcal{N}:e\in a_{i}\}\right|.$ When resource $e$ is used by $k$ agents, it generates reward $r_{e}(k)$ and incurs execution time $t_{e}(k)>0$. Hence, the reward and execution time of agent $i$ under profile $a$ are

$$R_{i}(a)=\sum_{e\in a_{i}}r_{e}(n_{e}(a)),\quad\text{and}\quad T_{i}(a)=\sum_{e\in a_{i}}t_{e}(n_{e}(a)).$$

We assume that every feasible action is nonempty, i.e., $a_{i}\neq\emptyset$ for all $a_{i}\in\mathcal{A}_{i}$. Hence, $T_{i}(a)>0$ for every agent $i$ and every feasible profile $a$. The corresponding reward-rate payoff is $J_{i}(a)=\frac{R_{i}(a)}{T_{i}(a)}.$ Thus, each agent seeks to maximize reward per unit execution time. We refer to the resulting game as a reward-rate congestion game.

The ratio form of $J_{i}$ does not, in general, admit an exact potential. Motivated by the Dinkelbach transformation, for each fixed $\rho\in\mathbb{R}$, the transformed payoff $Q_{i}^{\rho}(a)=R_{i}(a)-\rho T_{i}(a).$ Equivalently, $Q_{i}^{\rho}(a)=\sum_{e\in a_{i}}\left[r_{e}(n_{e}(a))-\rho t_{e}(n_{e}(a))\right].$ Define the transformed resource payoff $m_{e}^{\rho}(k):=r_{e}(k)-\rho t_{e}(k).$ Then $Q_{i}^{\rho}(a)=\sum_{e\in a_{i}}m_{e}^{\rho}(n_{e}(a)).$

Recall that a game is an exact potential game if every unilateral change in a player’s payoff equals the corresponding change in a common potential function [5]. For fixed $\rho$, define the Rosenthal-type potential

$$\Phi^{\rho}(a)=\sum_{e\in\mathcal{E}}\sum_{q=1}^{n_{e}(a)}m_{e}^{\rho}(q).$$

Equivalently, $\Phi^{\rho}(a)=\Phi_{R}(a)-\rho\Phi_{T}(a),$ where

$$\Phi_{R}(a)=\sum_{e\in\mathcal{E}}\sum_{q=1}^{n_{e}(a)}r_{e}(q),\qquad\Phi_{T}(a)=\sum_{e\in\mathcal{E}}\sum_{q=1}^{n_{e}(a)}t_{e}(q).$$

Proposition 1.

For every fixed $\rho\in\mathbb{R}$, the transformed game with payoffs $\{Q_{i}^{\rho}\}_{i\in\mathcal{N}}$ is an exact potential game with potential $\Phi^{\rho}$.

Proof.

Fix an agent $i$, an action profile $a=(a_{i},a_{-i})$, and an alternative action $a_{i}^{\prime}\in\mathcal{A}_{i}$. Let $a^{\prime}=(a_{i}^{\prime},a_{-i})$. Only resources that are added or removed by player $i$’s deviation can change their congestion levels. Define $S^{+}:=a_{i}^{\prime}\setminus a_{i}$ and $S^{-}:=a_{i}\setminus a_{i}^{\prime}$. For $e\in S^{+}$, the congestion level increases from $n_{e}(a)$ to $n_{e}(a)+1$. For $e\in S^{-}$, it decreases from $n_{e}(a)$ to $n_{e}(a)-1$. Therefore,

$$\Phi^{\rho}(a^{\prime})-\Phi^{\rho}(a)=\sum_{e\in S^{+}}m_{e}^{\rho}(n_{e}(a)+1)-\sum_{e\in S^{-}}m_{e}^{\rho}(n_{e}(a)).$$

Similarly,

$$Q_{i}^{\rho}(a^{\prime})-Q_{i}^{\rho}(a)=\sum_{e\in S^{+}}m_{e}^{\rho}(n_{e}(a)+1)-\sum_{e\in S^{-}}m_{e}^{\rho}(n_{e}(a)).$$

Hence,

$$\Phi^{\rho}(a^{\prime})-\Phi^{\rho}(a)=Q_{i}^{\rho}(a^{\prime})-Q_{i}^{\rho}(a),$$

which proves that $\Phi^{\rho}$ is an exact potential. ∎

Thus, although the original reward-rate payoffs $J_{i}=R_{i}/T_{i}$ are generally non-potential, the Dinkelbach-transformed payoffs $Q_{i}^{\rho}=R_{i}-\rho T_{i}$ define an exact potential game for every fixed $\rho$.

We now define the potential-level reward-rate problem associated with the transformed potential. Since every feasible action is nonempty and $t_{e}(k)>0$ for all $e\in\mathcal{E}$ and all $k\geq 1$, we have $\Phi_{T}(a)>0$ for all $a\in\mathcal{A}$. The potential-level reward-rate objective is

$$\rho_{\Phi}^{\star}=\max_{a\in\mathcal{A}}\frac{\Phi_{R}(a)}{\Phi_{T}(a)}.$$ (1)

By the Dinkelbach transformation [13], the optimal ratio is the unique zero of the associated parametric value function

$$\psi_{\Phi}(\rho)=\max_{a\in\mathcal{A}}\left\{\Phi_{R}(a)-\rho\Phi_{T}(a)\right\}=\max_{a\in\mathcal{A}}\Phi^{\rho}(a).$$

Then $\rho_{\Phi}^{\star}$ is the unique root of $\psi_{\Phi}$, i.e., $\psi_{\Phi}(\rho_{\Phi}^{\star})=0,$ and

$$\rho<\rho_{\Phi}^{\star}\Rightarrow\psi_{\Phi}(\rho)>0,\qquad\rho>\rho_{\Phi}^{\star}\Rightarrow\psi_{\Phi}(\rho)<0.$$

The corresponding potential-level Dinkelbach iteration is

$$a^{k}\in\arg\max_{a\in\mathcal{A}}\Phi^{\rho_{k}}(a),$$ (2)

followed by

$$\rho_{k+1}=\frac{\Phi_{R}(a^{k})}{\Phi_{T}(a^{k})}.$$ (3)

For each fixed $\rho_{k}$, the profile $a^{k}$ is a global maximizer of the exact potential $\Phi^{\rho_{k}}$. Hence, $a^{k}$ is a pure Nash equilibrium of the transformed game with payoffs $\{Q_{i}^{\rho_{k}}\}_{i\in\mathcal{N}}$. However, a pure Nash equilibrium of the transformed game need not be a global maximizer of $\Phi^{\rho_{k}}$. Therefore, convergence of the Dinkelbach iteration requires the inner potential maximization problem to be solved globally.

Theorem 1.

Suppose $\Phi_{T}(a)>0$ for all $a\in\mathcal{A}$, and initialize $\rho_{0}=\frac{\Phi_{R}(a^{0})}{\Phi_{T}(a^{0})}$ for some $a^{0}\in\mathcal{A}$. Suppose that, at each iteration, the inner problem (2) is solved globally. Then the sequence $\{\rho_{k}\}$ generated by (2)–(3) is nondecreasing and reaches $\rho_{\Phi}^{\star}$ in finitely many iterations, where $\rho_{\Phi}^{\star}$ is defined in (1). The corresponding profile $a^{\star}$ globally solves (1).

Proof.

Since $\rho_{0}=\Phi_{R}(a^{0})/\Phi_{T}(a^{0})$ is a feasible ratio, we have $\rho_{0}\leq\rho_{\Phi}^{\star}.$ At iteration $k$, because (2) is solved globally, we have $\psi_{\Phi}(\rho_{k})=\Phi_{R}(a^{k})-\rho_{k}\Phi_{T}(a^{k}).$ Using (3), we obtain $\rho_{k+1}=\frac{\Phi_{R}(a^{k})}{\Phi_{T}(a^{k})}=\rho_{k}+\frac{\psi_{\Phi}(\rho_{k})}{\Phi_{T}(a^{k})}.$ Since $\Phi_{T}(a^{k})>0$, if $\rho_{k}<\rho_{\Phi}^{\star}$, then the sign property of $\psi_{\Phi}$ gives $\psi_{\Phi}(\rho_{k})>0,$ and therefore $\rho_{k+1}>\rho_{k}.$ Moreover, $\rho_{k+1}$ is a feasible ratio, so $\rho_{k+1}\leq\rho_{\Phi}^{\star}.$ Hence, $\{\rho_{k}\}$ is nondecreasing and bounded above by $\rho_{\Phi}^{\star}$.

Since $\mathcal{A}$ is finite, the set of feasible ratios $\left\{\frac{\Phi_{R}(a)}{\Phi_{T}(a)}:a\in\mathcal{A}\right\}$ is finite. Therefore, the nondecreasing sequence $\{\rho_{k}\}$, whose elements belong to this finite set, must terminate in finite time. Let $\rho_{K}$ be its terminal value. If $\rho_{K}<\rho_{\Phi}^{\star}$, then the preceding argument implies $\rho_{K+1}>\rho_{K},$ which contradicts termination. Hence, $\rho_{K}=\rho_{\Phi}^{\star}.$ Finally, the terminal profile $a^{\star}:=a^{K}$ satisfies $\frac{\Phi_{R}(a^{\star})}{\Phi_{T}(a^{\star})}=\rho_{\Phi}^{\star},$ and therefore globally solves (1). ∎

Since the terminal profile $a^{\star}$ globally maximizes the exact potential $\Phi^{\rho_{\Phi}^{\star}}$, it is also a pure Nash equilibrium of the transformed game with payoffs $Q_{i}^{\rho_{\Phi}^{\star}}=R_{i}-\rho_{\Phi}^{\star}T_{i}.$ The next result gives a sufficient condition under which an equilibrium of the transformed game is also an equilibrium of the original reward-rate game.

Corollary 1.

Let $\bar{\rho}\in\mathbb{R}$, and let $a^{\star}$ be a pure Nash equilibrium of the transformed game with payoffs $Q_{i}^{\bar{\rho}}(a)=R_{i}(a)-\bar{\rho}T_{i}(a).$ If $\frac{R_{i}(a^{\star})}{T_{i}(a^{\star})}=\bar{\rho},\;\forall i\in\mathcal{N},$ then $a^{\star}$ is a pure Nash equilibrium of the direct reward-rate game with payoffs $J_{i}(a)=\frac{R_{i}(a)}{T_{i}(a)}.$

Proof.

Since $a^{\star}$ is a pure Nash equilibrium of the transformed game, for every player $i$ and every unilateral deviation $a_{i}^{\prime}\in\mathcal{A}_{i}$,

$$R_{i}(a^{\star})-\bar{\rho}T_{i}(a^{\star})\geq R_{i}(a_{i}^{\prime},a_{-i}^{\star})-\bar{\rho}T_{i}(a_{i}^{\prime},a_{-i}^{\star}).$$

By the assumed common-rate condition, $R_{i}(a^{\star})-\bar{\rho}T_{i}(a^{\star})=0.$ Therefore, $R_{i}(a_{i}^{\prime},a_{-i}^{\star})-\bar{\rho}T_{i}(a_{i}^{\prime},a_{-i}^{\star})\leq 0.$ Since $T_{i}(a_{i}^{\prime},a_{-i}^{\star})>0$, it follows that $\frac{R_{i}(a_{i}^{\prime},a_{-i}^{\star})}{T_{i}(a_{i}^{\prime},a_{-i}^{\star})}\leq\bar{\rho}.$ Using $\bar{\rho}=\frac{R_{i}(a^{\star})}{T_{i}(a^{\star})}$, we get $J_{i}(a_{i}^{\prime},a_{-i}^{\star})\leq J_{i}(a^{\star}),\;\forall a_{i}^{\prime}\in\mathcal{A}_{i}.$ Thus no player can improve its direct reward rate by a unilateral deviation, and $a^{\star}$ is a pure Nash equilibrium of the direct reward-rate game. ∎

Thus, an equilibrium of the transformed game is not, in general, a Nash equilibrium of the direct reward-rate game; Corollary 1 provides a sufficient condition under which the same profile is a Nash equilibrium of both games.

III Social-Reward-Rate Alignment via Externality Corrections

The potential-level reward-rate objective in (1) does not generally coincide with the social reward-rate objective. Indeed, the aggregate social reward and aggregate social execution time are

$$SW_{R}(a) =\sum_{i=1}^{N}R_{i}(a)=\sum_{e\in\mathcal{E}}n_{e}(a)r_{e}(n_{e}(a)),\quad\text{and}$$

$$SW_{T}(a) =\sum_{i=1}^{N}T_{i}(a)=\sum_{e\in\mathcal{E}}n_{e}(a)t_{e}(n_{e}(a)).$$

In general, $\Phi_{R}(a)\neq SW_{R}(a),\;\Phi_{T}(a)\neq SW_{T}(a).$ Thus, optimizing the potential-level ratio $\frac{\Phi_{R}(a)}{\Phi_{T}(a)}$ does not necessarily optimize the social reward rate $\frac{SW_{R}(a)}{SW_{T}(a)}.$

To align transformed individual incentives with the social reward-rate objective, we assign each resource its marginal contribution to aggregate reward and execution time. Specifically, define

$$\widetilde{r}_{e}(k) :=kr_{e}(k)-(k-1)r_{e}(k-1),\quad\text{and}$$

$$\widetilde{t}_{e}(k) :=kt_{e}(k)-(k-1)t_{e}(k-1),$$

with the convention $r_{e}(0)=0$ and $t_{e}(0)=0.$ The corrected payoff of agent $i$ is $\widetilde{R}_{i}(a)=\sum_{e\in a_{i}}\widetilde{r}_{e}(n_{e}(a))$ and $\widetilde{T}_{i}(a)=\sum_{e\in a_{i}}\widetilde{t}_{e}(n_{e}(a)).$ For fixed $\rho$, define the corrected transformed payoff $\widetilde{Q}_{i}^{\rho}(a)=\widetilde{R}_{i}(a)-\rho\widetilde{T}_{i}(a).$

Theorem 2.

For every fixed $\rho$, the corrected transformed game with payoffs $\{\widetilde{Q}_{i}^{\rho}\}_{i\in\mathcal{N}}$ is an exact potential game with potential $\widetilde{\Phi}^{\rho}(a)=SW_{R}(a)-\rho SW_{T}(a).$

Proof.

By construction,

$$\sum_{q=1}^{k}\widetilde{r}_{e}(q)=\sum_{q=1}^{k}\bigl[qr_{e}(q)-(q-1)r_{e}(q-1)\bigr]=kr_{e}(k),$$

and similarly, $\sum_{q=1}^{k}\widetilde{t}_{e}(q)=kt_{e}(k).$

Hence, the Rosenthal reward and execution-time potentials of the corrected game satisfy

$$\widetilde{\Phi}_{R}(a) =\sum_{e\in\mathcal{E}}n_{e}(a)r_{e}(n_{e}(a))=\mathrm{SW}_{R}(a),\quad\text{and}$$

$$\widetilde{\Phi}_{T}(a) =\sum_{e\in\mathcal{E}}n_{e}(a)t_{e}(n_{e}(a))=\mathrm{SW}_{T}(a).$$

Therefore,

$$\widetilde{\Phi}^{\rho}(a)=\widetilde{\Phi}_{R}(a)-\rho\widetilde{\Phi}_{T}(a)=\mathrm{SW}_{R}(a)-\rho\mathrm{SW}_{T}(a).$$

The exact-potential property then follows from the same unilateral-deviation argument as in Proposition 1. ∎

Consequently, since $SW_{T}(a)>0$ for all feasible profiles, the social reward-rate problem

$$\rho_{SW}^{\star}=\max_{a\in\mathcal{A}}\frac{SW_{R}(a)}{SW_{T}(a)}$$

admits the same potential-level Dinkelbach construction as in Section II. Under global solution of each inner problem,

$$a^{k}\in\arg\max_{a\in\mathcal{A}}\left\{SW_{R}(a)-\rho_{k}SW_{T}(a)\right\},\quad\rho_{k+1}=\frac{SW_{R}(a^{k})}{SW_{T}(a^{k})}$$

reaches the optimal social reward rate in finitely many iterations.

IV Task Allocation as a Singleton Congestion Game

Task allocation[14] provides a natural setting for the proposed reward-rate congestion-game framework. Let $\mathcal{T}=\{1,\dots,M\}$ denote a finite set of tasks, and suppose each agent selects one task. The resulting game is a singleton congestion game in which tasks are the resources. For a task $j$, let

$$S_{j}(a):=\{i\in\mathcal{N}:a_{i}=j\},\quad\text{and}\quad n_{j}(a):=|S_{j}(a)|$$

denote the assigned coalition and its size.

Suppose that task $j$ generates individual reward $r_{j}(k)$ and individual execution time $t_{j}(k)>0$ when $k$ agents are assigned to it. Then the reward and execution time of agent $i$, with $a_{i}=j$, are $R_{i}(a)=r_{j}(n_{j}(a)),$ and $T_{i}(a)=t_{j}(n_{j}(a)).$ The corresponding fixed-parameter transformed payoff is

$$Q_{i}^{\rho}(a)=r_{a_{i}}(n_{a_{i}}(a))-\rho t_{a_{i}}(n_{a_{i}}(a)).$$

By the previous results, the transformed task-allocation game is an exact potential game with potential

$$\Phi^{\rho}(a)=\sum_{j=1}^{M}\sum_{q=1}^{n_{j}(a)}\left[r_{j}(q)-\rho t_{j}(q)\right].$$

In the language of anonymous hedonic games [15], the coalition assigned to task $j$ is $S_{j}(a)$, and each agent’s payoff depends only on the selected task and the size of its coalition. Thus, for both the direct reward-rate payoff and the fixed-parameter transformed payoff, the task-allocation model is equivalently a singleton congestion game. Nash stability of the allocation is therefore equivalent to pure Nash equilibrium: no agent can improve its payoff by unilaterally switching tasks.

V Continuous-Time Replicator–Dinkelbach Dynamics

We now consider a continuous population formulation of the social reward-rate problem. Let $x=(x_{1},\dots,x_{M})\in\Delta_{M},$ where $x_{j}$ denotes the fraction of the population assigned to task $j$. Let $B_{j}(x_{j})$ and $H_{j}(x_{j})$ denote the aggregate reward and execution time associated with task $j$. Define

$$R(x)=\sum_{j=1}^{M}B_{j}(x_{j}),\qquad T(x)=\sum_{j=1}^{M}H_{j}(x_{j}),$$

and assume $T(x)>0$ for all $x\in\Delta_{M}$. The continuous reward-rate problem is

$$\rho^{\star}=\max_{x\in\Delta_{M}}\frac{R(x)}{T(x)}.$$ (4)

For a fixed $\rho$, define the Dinkelbach-transformed objective

$$W^{\rho}(x)=R(x)-\rho T(x).$$

The marginal transformed payoff of task $j$ is

$$u_{j}^{\rho}(x):=\frac{\partial W^{\rho}(x)}{\partial x_{j}}=B_{j}^{\prime}(x_{j})-\rho H_{j}^{\prime}(x_{j}).$$

These marginal transformed payoffs are the population analogue of the externality-corrected payoffs in Section III. Let $\bar{u}^{\rho}(x)=\sum_{j=1}^{M}x_{j}u_{j}^{\rho}(x).$ For fixed $\rho$, consider the replicator dynamics [16, 17]

$$\dot{x}_{j}=x_{j}\left(u_{j}^{\rho}(x)-\bar{u}^{\rho}(x)\right),\qquad j=1,\dots,M.$$ (5)

The dynamics (5) is the inner, or fast, dynamics for the Dinkelbach-transformed objective with $\rho$ fixed. To solve the reward-rate problem (4), we couple this inner dynamics with a slow Dinkelbach update for $\rho$:

$$\dot{x}_{j} =x_{j}\left(u_{j}^{\rho}(x)-\bar{u}^{\rho}(x)\right),\qquad j=1,\dots,M,$$ (6)

$$\dot{\rho} =\varepsilon\left(\frac{R(x)}{T(x)}-\rho\right),\qquad 0<\varepsilon\ll 1.$$

The small parameter $\varepsilon$ separates the time scales: $x$ evolves on the fast time scale, while $\rho$ evolves slowly according to the currently achieved reward rate $R(x)/T(x)$. We first analyze the frozen-$\rho$ inner dynamics (5), and then study the reduced slow dynamics induced by (6).

Assumption 1.

For every $j$, $B_{j}$ and $H_{j}$ are thrice continuously differentiable on an open interval containing $[0,1]$. Moreover, there exists $\mu>0$ such that $B_{j}^{\prime\prime}(s)\leq-\mu$ and $H_{j}^{\prime\prime}(s)\geq 0$, for each $s\in[0,1]$, and $T(x)>0$, for every $x\in\Delta_{M}$.

The above assumption implies $W^{\rho}$ is uniformly strongly concave and has a unique maximizer.

Assumption 2.

There exists a compact interval $\mathcal{I}\subset\mathbb{R}_{\geq 0}$ containing $\rho^{\star}$ such that $x^{\star}(\rho)\in\operatorname{int}(\Delta_{M})$, for every $\rho\in\mathcal{I}$.

Lemma 1.

Under Assumptions 1 and 2, the optimizer map $\rho\mapsto x^{\star}(\rho)$ is twice continuously differentiable on $\mathcal{I}$.

Proof.

Since $x^{\star}(\rho)\in\operatorname{int}(\Delta_{M})$, the KKT conditions for maximizing $W^{\rho}$ over $\Delta_{M}$ are

$$B_{j}^{\prime}(x_{j}^{\star})-\rho H_{j}^{\prime}(x_{j}^{\star})=\lambda,\qquad j=1,\ldots,M,$$

together with $\mathbf{1}^{\top}x^{\star}=1$. Define

$$F(x,\lambda,\rho)=\begin{bmatrix}B_{1}^{\prime}(x_{1})-\rho H_{1}^{\prime}(x_{1})-\lambda\\ \vdots\\ B_{M}^{\prime}(x_{M})-\rho H_{M}^{\prime}(x_{M})-\lambda\\ \mathbf{1}^{\top}x-1\end{bmatrix}.$$

Its Jacobian with respect to $(x,\lambda)$ is $D_{(x,\lambda)}F=\begin{bmatrix}D&-\mathbf{1}\\ \mathbf{1}^{\top}&0\end{bmatrix}$, where $D=\operatorname{diag}\left(B_{j}^{\prime\prime}(x_{j})-\rho H_{j}^{\prime\prime}(x_{j})\right)$.

By Assumption 1, $D\preceq-\mu I$. Consider a vector $\begin{bmatrix}v^{\top}\\ \alpha\end{bmatrix}^{\top}$. Suppose

$$Dv-\alpha\mathbf{1}=0,\quad\text{and}\quad\mathbf{1}^{\top}v=0.$$

Multiplying the first equality by $v^{\top}$ gives $v^{\top}Dv=\alpha\mathbf{1}^{\top}v=0$. Since $D$ is negative definite, $v=0$, and consequently $\alpha=0$. Hence, $D_{(x,\lambda)}F$ is nonsingular.

The implicit-function theorem therefore implies that $(x^{\star}(\rho),\lambda^{\star}(\rho))$ is twice continuously differentiable locally in $\rho$. Since the maximizer is unique for every $\rho\in\mathcal{I}$, these local representations define a twice continuously differentiable optimizer map $x^{\star}(\rho)$ on $\mathcal{I}$. ∎

It is well known that the simplex $\Delta_{M}$ is forward invariant under replicator dynamics [17]. Moreover, for fixed $\rho$,

$$\dot{W}^{\rho}(x(t))=\sum_{j=1}^{M}x_{j}\left(u_{j}^{\rho}(x)-\bar{u}^{\rho}(x)\right)^{2}\geq 0,$$

so the transformed objective is nondecreasing along its trajectories. The next result characterizes the stability of the frozen-$\rho$ fast subsystem.

Theorem 3.

Fix $\rho\geq 0$. Under Assumption 1, every solution of the frozen-$\rho$ replicator dynamics (5) with an interior initial condition satisfies $x(t)\longrightarrow x^{\star}(\rho)$. Thus, $x^{\star}(\rho)$ is globally asymptotically attractive relative to $\operatorname{int}(\Delta_{M})$.

Moreover, under Assumption 2, $x^{\star}(\rho)$ is locally exponentially stable for every $\rho\in\mathcal{I}$, uniformly in $\rho$.

Proof.

Fix $\rho$ and write $x^{\star}=x^{\star}(\rho)$. Consider

$$V(x)=\sum_{j=1}^{M}x_{j}^{\star}\log\frac{x_{j}^{\star}}{x_{j}}.$$

Since $x(0)\in\operatorname{int}(\Delta_{M})$, the replicator dynamics preserves positivity, so $x_{j}(t)>0$ for all $j$ and all $t\geq 0$. Hence the logarithmic terms in $V(x(t))$ are finite, and $V$ is well-defined along solutions. Since $V$ is the Kullback–Leibler divergence from $x^{\star}$ to $x$, $V(x)\geq 0,$ with equality if and only if $x=x^{\star}$. Along (5),

$$\dot{V}=-\sum_{j=1}^{M}x_{j}^{\star}\frac{\dot{x}_{j}}{x_{j}}=-\sum_{j=1}^{M}x_{j}^{\star}\left(u_{j}^{\rho}(x)-\bar{u}^{\rho}(x)\right).$$

Since $\sum_{j=1}^{M}x_{j}^{\star}=1$ and $\bar{u}^{\rho}(x)=\sum_{j=1}^{M}x_{j}u_{j}^{\rho}(x),$ we obtain

$$\dot{V}=-\sum_{j=1}^{M}x_{j}^{\star}u_{j}^{\rho}(x)+\sum_{j=1}^{M}x_{j}u_{j}^{\rho}(x).$$

Therefore, $\dot{V}=-\nabla W^{\rho}(x)^{\top}(x^{\star}-x).$ By concavity of $W^{\rho}$,

$$W^{\rho}(x^{\star})-W^{\rho}(x)\leq\nabla W^{\rho}(x)^{\top}(x^{\star}-x).$$

Since $x^{\star}$ maximizes $W^{\rho}$, the left-hand side is nonnegative. Hence,

$$\dot{V}\leq-\left(W^{\rho}(x^{\star})-W^{\rho}(x)\right)\leq 0.$$

Define the optimality gap $g(t):=W^{\rho}(x^{\star})-W^{\rho}(x(t))\geq 0.$ From $\dot{V}\leq-g(t),$ it follows that

$$\int_{0}^{T}g(t)\,dt\leq V(x(0))-V(x(T))\leq V(x(0)).$$

Therefore, $g(t)$ is integrable. Moreover, $g(t)$ is uniformly continuous. Indeed, $\dot{g}(t)=-\nabla W^{\rho}(x(t))^{\top}\dot{x}(t).$ Since $x(t)\in\Delta_{M}$, and $\Delta_{M}$ is compact, $\nabla W^{\rho}$ is bounded on $\Delta_{M}$. Also, the vector field in (5) is continuous on $\Delta_{M}$, and hence $\dot{x}(t)$ is bounded. Therefore, $\dot{g}(t)$ is bounded, which implies that $g(t)$ is uniformly continuous. Hence, by Barbalat’s lemma, $g(t)\to 0.$ Thus, $W^{\rho}(x(t))\to W^{\rho}(x^{\star}).$ Since $W^{\rho}$ is strictly concave, $x^{\star}$ is the unique maximizer, and therefore $x(t)\to x^{\star}.$

To establish the local exponential property, let $\rho\in\mathcal{I}$. Uniform strong concavity gives

$$W^{\rho}(x^{\star}(\rho))-W^{\rho}(x)\geq\frac{\mu}{2}\|x-x^{\star}(\rho)\|^{2}.$$

Hence, for the KL Lyapunov function used above,

$$\dot{V}\leq-\frac{\mu}{2}\|x-x^{\star}(\rho)\|^{2}.$$

By Lemma 1, $x^{\star}(\rho)$ is continuous in $\rho$. Since $\mathcal{I}$ is compact and $x^{\star}(\rho)$ is interior, the equilibrium manifold is uniformly separated from the boundary of the simplex. Consequently, in a uniform neighborhood of this manifold,

$$c_{1}\|x-x^{\star}(\rho)\|^{2}\leq V(x)\leq c_{2}\|x-x^{\star}(\rho)\|^{2},$$

where $c_{1},c_{2}>0$ are independent of $\rho$. Therefore, $x^{\star}(\rho)$ is locally exponentially stable uniformly over $\rho\in\mathcal{I}$. ∎

Theorem 3 therefore provides the boundary-layer stability property required for the two-time-scale analysis. We next consider the reduced slow dynamics obtained by restricting the fast state to the optimizer manifold $x=x^{\star}(\rho)$. Substituting $x=x^{\star}(\rho)$ into (6) gives

$$\dot{\rho}=\varepsilon\bigl(h(\rho)-\rho\bigr),$$ (7)

where $h(\rho):=\frac{R(x^{\star}(\rho))}{T(x^{\star}(\rho))}.$

Theorem 4.

Under Assumptions 1 and 2, the optimal value $\rho^{\star}$ in (4) is globally asymptotically stable and locally exponentially stable for the reduced dynamics (7).

Proof.

Define

$$\psi(\rho)=\max_{x\in\Delta_{M}}\{R(x)-\rho T(x)\}.$$

Since $x^{\star}(\rho)$ maximizes $R(x)-\rho T(x)$,

$$\psi(\rho)=R(x^{\star}(\rho))-\rho T(x^{\star}(\rho))=T(x^{\star}(\rho))(h(\rho)-\rho).$$

Because $T(x^{\star}(\rho))>0$, the sign of $h(\rho)-\rho$ is the same as the sign of $\psi(\rho)$. By the Dinkelbach sign property,

$$\rho<\rho^{\star}\Rightarrow h(\rho)-\rho>0,\qquad\rho>\rho^{\star}\Rightarrow h(\rho)-\rho<0.$$

Consider $V_{\rho}(\rho)=\frac{1}{2}(\rho-\rho^{\star})^{2}.$ Along (7),

$$\dot{V}_{\rho}=\varepsilon(\rho-\rho^{\star})\bigl(h(\rho)-\rho\bigr).$$

If $\rho>\rho^{\star}$, then $h(\rho)-\rho<0$, and hence $\dot{V}_{\rho}<0$. If $\rho<\rho^{\star}$, then $h(\rho)-\rho>0$, and again $\dot{V}_{\rho}<0$. Therefore, $\dot{V}_{\rho}<0,\;\forall\rho\neq\rho^{\star}.$ Thus, $\rho^{\star}$ is globally asymptotically stable for the reduced slow dynamics.

It remains to establish local exponential stability. Define

$$\psi(\rho)=W^{\rho}(x^{\star}(\rho))=T(x^{\star}(\rho))(h(\rho)-\rho).$$

Differentiating and using the KKT condition $\nabla W^{\rho}(x^{\star})=\lambda\mathbf{1}$ gives

$$\psi^{\prime}(\rho) =\nabla W^{\rho}(x^{\star})^{\top}{x^{\star}}^{\prime}(\rho)-T(x^{\star}(\rho))$$

$$=\lambda\mathbf{1}^{\top}{x^{\star}}^{\prime}(\rho)-T(x^{\star}(\rho))=-T(x^{\star}(\rho)),$$

where $\mathbf{1}^{\top}{x^{\star}}^{\prime}(\rho)=0$ follows from $\mathbf{1}^{\top}x^{\star}(\rho)=1$.

Since $\psi(\rho^{\star})=0$,

$$\left.\frac{d}{d\rho}(h(\rho)-\rho)\right|_{\rho=\rho^{\star}}=\frac{\psi^{\prime}(\rho^{\star})}{T(x^{\star}(\rho^{\star}))}=-1.$$

Therefore, the linearization of the reduced dynamics at $\rho^{\star}$ is $\dot{\widetilde{\rho}}=-\varepsilon\widetilde{\rho},$ and $\rho^{\star}$ is locally exponentially stable. ∎

The preceding results provide the fast- and slow-subsystem stability properties required for the singular-perturbation analysis.

Theorem 5 (Two-time-scale exponential stability).

Suppose Assumptions 1 and 2 hold. Then there exists $\varepsilon^{\star}>0$ such that, for every $0<\varepsilon<\varepsilon^{\star}$, $\bigl(x^{\star}(\rho^{\star}),\rho^{\star}\bigr)$ is a locally exponentially stable equilibrium of the coupled replicator–Dinkelbach dynamics (6).

Proof.

Using the slow time $s=\varepsilon t$, the coupled dynamics has the standard singularly perturbed form with slow variable $\rho$ and fast variable $x$. By Lemma 1, the isolated equilibrium $x^{\star}(\rho)$ of the fast subsystem depends smoothly on $\rho$. Under Assumption 1, the system functions and their required derivatives are locally bounded.

By Theorem 4, the equilibrium $\rho^{\star}$ of the reduced system is locally exponentially stable. By Theorem 3, the equilibrium $x^{\star}(\rho)$ of the boundary-layer system is locally exponentially stable uniformly in $\rho\in\mathcal{I}$. Therefore, the hypotheses of [18, Thm. 11.4] are satisfied, and the result follows. ∎

Thus, for sufficiently small $\varepsilon$, time-scale separation preserves the local exponential stability of the reward-rate optimum: the fast replicator dynamics tracks the optimizer $x^{\star}(\rho)$, while the slow Dinkelbach update drives $\rho$ toward $\rho^{\star}$.

Remark 1.

The analysis is not specific to replicator dynamics. The same singular-perturbation argument applies to any sufficiently smooth fast learning dynamics whose equilibrium $x^{\star}(\rho)$ solves the fixed-$\rho$ potential problem and is uniformly locally exponentially stable in $\rho\in\mathcal{I}$.

VI Numerical Example

We illustrate the proposed replicator–Dinkelbach dynamics on a continuous task-allocation problem with three tasks: victim search, damage inspection, and communication relay/mapping. For each task $j\in\{1,2,3\}$, let

$$B_{j}(x_{j})=\alpha_{j}(1-e^{-\beta_{j}x_{j}}),\qquad H_{j}(x_{j})=c_{j}x_{j}+d_{j}x_{j}^{2},$$

where $x_{j}$ is the fraction of the population assigned to task $j$. The function $B_{j}$ models saturating task reward, while $H_{j}$ models increasing execution-time burden due to congestion. We use the parameters

$$\begin{array}[]{c|cccc}\text{task }j&\alpha_{j}&\beta_{j}&c_{j}&d_{j}\\ \hline\cr 1&10&4.0&1.2&3.0\\ 2&7&2.5&0.8&1.2\\ 3&5.5&1.5&0.5&0.5\end{array}$$

and define $R(x)=\sum_{j=1}^{3}B_{j}(x_{j}),\;T(x)=\sum_{j=1}^{3}H_{j}(x_{j}).$ The objective is $\rho^{\star}=\max_{x\in\Delta_{3}}{R(x)}/{T(x)}.$ For a fixed $\rho$, the marginal transformed payoff is

$$u_{j}^{\rho}(x)=\alpha_{j}\beta_{j}e^{-\beta_{j}x_{j}}-\rho(c_{j}+2d_{j}x_{j}).$$

We simulate the replicator–Dinkelbach dynamics (6) with $\varepsilon=0.05$. As a reference solution, we first compute the optimal reward-rate pair $(x^{\star},\rho^{\star})$ using the static Dinkelbach iteration. Starting from $\rho_{0}\geq 0$, each iteration solves $x^{k}\in\arg\max_{x\in\Delta_{3}}\left\{R(x)-\rho_{k}T(x)\right\},$ and updates $\rho_{k+1}={R(x^{k})}/{T(x^{k})}.$ Since $R$ is strictly concave and $T$ is strictly convex, $R-\rho_{k}T$ is strictly concave for every $\rho_{k}\geq 0$. Hence, the inner problem has a unique global maximizer over $\Delta_{3}$. For the parameters above, the iteration gives $x^{\star}\approx(0.1920,\;0.2797,\;0.5282)^{\top},\;\rho^{\star}\approx 11.1941.$

Figure 1 shows that, from different initial conditions, all allocation trajectories converge to the same optimizer $x^{\star}$, while the corresponding Dinkelbach variables converge to $\rho^{\star}$. This illustrates the convergence behavior established by the analysis.

  

Fig. 1: Convergence of the replicator–Dinkelbach dynamics.

VII Conclusion

This paper developed a Dinkelbach-based framework for reward-rate congestion games. The fixed-parameter transformation recovers an exact potential structure, yielding finite termination of the potential-level Dinkelbach iteration under global inner maximization. Marginal externality corrections align the potential with the social reward-rate objective. For continuous task allocation, the proposed replicator–Dinkelbach dynamics yields convergence of the fast and reduced subsystems and local exponential stability of the coupled equilibrium for sufficiently slow Dinkelbach updates. Future work will extend the framework to nonconcave transformed potentials and to discrete-time learning dynamics with global convergence guarantees.

VIII References