CCLab: Adversarial Testing of Learning- and Non-Learning-Based Congestion Controllers
Extracted source text · arXiv 2605.21915
\documentclass[conference]{IEEEtran}
% Some very useful LaTeX packages include:
% (uncomment the ones you want to load)
\usepackage{comment}
\usepackage{soul}
\usepackage{spverbatim}
\usepackage{listings}
\usepackage{mdframed}
\usepackage{fancyvrb}
\usepackage{multirow}
\usepackage{minted}
\usepackage{multicol}
\usepackage{tcolorbox}
\usepackage{colortbl}
\usepackage{pifont}
\usepackage{makecell}
\usepackage{graphicx}
\usepackage{caption}
\usepackage{subcaption}
\usepackage{booktabs}
\usepackage{alltt}
\usepackage{rotating}
\usepackage{array}
\usepackage{multirow}
\usepackage{amsfonts}
\usepackage{amsmath}
\usepackage{xcolor}
\usepackage{siunitx}
\usepackage{algorithm}
\usepackage{algpseudocode}
\usepackage[normalem]{ulem}
\newcounter{remark}
\definecolor{navyblue}{rgb}{0.0, 0.0, 0.5}
\definecolor{darkgreen}{rgb}{0.0, 0.4, 0.0}
\newcommand{\cut}[1]{\textcolor{red}{\sout{#1}}}
\newcommand{\gang}[1]{{\color{blue} {\bf Gang:} #1}}
\newcommand{\shehab}[1]{{\color{cyan} {\bf Shehab:} #1}}
\newcommand{\zhi}[1]{{\color{orange} {\bf Zhi:} #1}}
\newcommand{\nic}[1]{{\color{teal} {\bf Nicholas:} #1}}
\newcommand{\sysname}{CCLab}
\newcommand{\insight}[1]{{\color{purple} #1}}
\newcommand{\revise}[1]{{\color{red} #1}}
\newcommand{\para}[1]{{\vspace{3pt} \bf \noindent #1 \hspace{8pt}}}
% *** GRAPHICS RELATED PACKAGES ***
%
\ifCLASSINFOpdf
\else
\fi
\pagestyle{plain}
\hyphenation{op-tical net-works semi-conduc-tor}
\begin{document}
%
\title{\sysname: Adversarial Testing of Learning- and Non-Learning-Based Congestion Controllers}
\author{\IEEEauthorblockN{Zhi Chen}
\IEEEauthorblockA{UIUC\\
zhic4@illinois.edu}
\and
\IEEEauthorblockN{Shehab Sarar Ahmed}
\IEEEauthorblockA{UIUC \& BUET\\
shehaba2@illinois.edu}
\and
\IEEEauthorblockN{Chenkai Wang}
\IEEEauthorblockA{UIUC\\
chenkai3@illinois.edu}
\and
\IEEEauthorblockN{Brighten Godfrey}
\IEEEauthorblockA{UIUC\\
pbg@illinois.edu}
\and
\IEEEauthorblockN{Gang Wang}
\IEEEauthorblockA{UIUC\\
gangw@illinois.edu}}
\maketitle
\begin{abstract}
Congestion controllers (CCs) are critical to network performance, and yet their robustness under adverse conditions remains insufficiently understood. While recent learning-based CCs have demonstrated strong performance in controlled environments, it is unclear how they compare to traditional CCs when controllers' input signals are corrupted or when environmental conditions become systematically challenging.
In this paper, we introduce \sysname{}, an adversarial testing framework for systematically evaluating the robustness of both learning-based and non-learning-based CCs. \sysname{} includes a reinforcement learning (RL)-based adversarial agent that operates in a closed loop with the congestion control policy, generating bounded perturbations either on input signals (feature-level) or on external network conditions (environment-level), while preserving realism through explicit constraints. Using this framework, we compare learning-based CCs with non-learning-based CCs under both feature-level and environment-level adversarial conditions. While both types of CCs suffer from performance degradation under adversarial testing, we find that learning-based CCs, in general, are more robust than traditional human-designed algorithms. Finally, we show that our adversarial traces can be used to train more robust CCs that outperform existing learning-based CCs under both challenging and normal conditions.
\end{abstract}
% no keywords
\IEEEpeerreviewmaketitle
\input{1-Introduction}
\input{2-RelatedWorks}
\input{3-Framework}
% \input{4-ExperimentalDesign}
\input{5-FeatureLevel}
\input{6-EnvironmentLevel}
\input{7-AdversarialTraining}
\input{8-Discussion}
\input{9-Ethics}
% \begin{small}
\bibliographystyle{IEEEtran}
\bibliography{sample}
% \end{small}
\input{10-Appendix}
\end{document}
\section{Introduction}
\label{sec:intro}
% \zhi{appendix: add soly perturbing link delay experiment}
% \zhi{5/4: related work section: add more stress-testing related papers}
% \zhi{write ethics section}
% \zhi{write discussion sections to compare with Giacomoni and Mazilu}
% \zhi{go back to related work and double check arguments on how we are different.}
% \gang{Very good intro overall. Need to add citations to different places.}
% \gang{add citations}
% \gang{Changing all ``C3'' to ``Canopy'' in the paper, including figures and tables.}
% \nic{minor but I believe I'm usually seeing CC=congestion control or controller, and
% ``Learning-based congestion controllers" are referred as LCCs, CC feels a little bit odd to me. Especially when we are testing specific impl. not the alg. themselves, I would consider drop the ``A''}
% \shehab{Can you give example of network signals?}
% \shehab{Can you give example of action?}
Congestion controllers (CCs) play a central role in determining the performance and stability of modern networks. At the high level, CCs map the observed network signals (e.g., round-trip time, pacing rate, and estimated delay) to control actions (e.g., adjusting the size of the congestion window) to effectively utilize network resources while avoiding congestion collapse. Over multiple decades, CCs were manually designed by human experts using analytical models and heuristic control logic.
% such as loss-based and delay-based approaches.
More recently, advances in machine learning have led to the emergence of {\em learning-based} congestion controllers~\cite{abbasloo2020classic,yang2024canopy,yen2023computers,jay2019deep,dong2018pcc} which have demonstrated promising performance in controlled environments.
% While learning-based CCs often achieve high throughput and low latency under benchmark conditions, their deployment in real networks raises an important yet underexplored question: how robust are learning-based congestion control policies compared to traditional, non-learning-based algorithms when operating under adverse or unexpected conditions? Existing evaluations predominantly focus on average-case performance, fixed traces, or benign variability, leaving a gap in our understanding of how different CCs behave when network conditions report misleading signals or evolve in worst-case—but still plausible—ways.
% Evaluating robustness in congestion control is fundamentally challenging. Learning-based and non-learning-based CCs differ not only in their internal logic, but also in the signals they consume and the assumptions they make about the network environment. As a result, direct comparison under identical perturbations is often infeasible. Moreover, many existing stress-testing approaches evaluate CCs using the same random perturbations or pre-defined synthetic traces across all policies, without adapting the test conditions to the behavior of the specific congestion control algorithm under evaluation. While such testing is useful for characterizing average-case behavior, it may fail to reveal systematic weaknesses that emerge only under policy-specific, worst-case conditions.
% \gang{I think most existing papers still use the term ``learning-based'' CC, instead of ``learned'', we should stay consistent with existing literature.}
% \gang{double check existing papers, and add citations}
% \gang{add citations}
% While learning-based CCs often achieve high throughput and low latency under controlled benchmark testing, one key question needs to be answered before pushing for a wide deployment in real networks: {\em how robust are learning-based congestion controllers compared to traditional, non-learning-based controllers under adverse or unexpected conditions? }
% Evaluations in prior works~\cite{abbasloo2020classic,yang2024canopy,giacomoni2024reinforcement, mazilu2025learning} provide useful insights into typical behavior, but largely focus on performance under relatively benign conditions. As a result, it remains unclear how different CCs behave when network dynamics
% become misleading, highly challenging, or systematically unfavorable.
% Answering this question is difficult for a number of reasons. Learning-based and non-learning-based CCs differ not only in their internal logic, but also in the signals they consume and the assumptions they make about the network environment. Consequently, direct comparison under identical perturbations is often infeasible. Existing stress-testing approaches partially address this issue, but many still rely on shared or predefined evaluation scenarios~\cite{abbasloo2020classic,yang2024canopy,giacomoni2024reinforcement, mazilu2025learning} that are applied uniformly across controllers, without adapting the test conditions to the controller under evaluation.
% \shehab{I couldn't understand what you wanted to convey in this sentence. I think the sentence should clearly state the limitation. You are saying first that it partially addresses the issue, and then you said but many .... So some are fully addressing the issue? Also, what do you mean by shared evaluation scenarios, what is meant by applied uniformly across policies, what is a test condition? Another word came here that is controller, what is a controller?}
% Such designs are useful for broad benchmarking, but they may fail to expose systematic weaknesses that arise only under controller-specific and worst-case, yet still plausible, conditions.
While learning-based CCs often achieve high throughput and low latency under controlled benchmark testing, one key question remains before their wide deployment in real networks: {\em how robust are learning-based congestion controllers compared to traditional, non-learning-based controllers under adverse or unexpected conditions?} Prior evaluations~\cite{abbasloo2020classic,yang2024canopy,giacomoni2024reinforcement, mazilu2025learning} provide useful insights into typical behavior, but largely focus on performance under relatively benign conditions. As a result, it remains unclear how different CCs behave when network dynamics become misleading, highly challenging, or systematically unfavorable.
Answering this question is further complicated by the fundamental differences between learning-based and non-learning-based CCs in terms of their internal control logic and the input signals they rely on.
% and their assumptions about the network environment \zhi{maybe remove assumptions about network environment, we only need to emphasize different input signals here}.
Existing stress-testing approaches~\cite{giacomoni2024reinforcement, mazilu2025learning} often rely on predefined (manually created) evaluation scenarios where identical network conditions are applied to all CCs without adapting to the specific characteristics of each controller. This limits the ability to expose weaknesses that arise only under controller-specific worst-case conditions.
Other works~\cite{ahmed26advnet,ray2022cc,robust-hotnet19} move beyond fixed scenarios by automatically generating network traces that degrade the performance of a given CC.
However, these works still focus on traditional, non-learning-based CCs without considering/comparing against {\em learning-based CCs}. Also, they all focus on modifying the network environmental conditions without evaluating the impact of the input signals to CCs.
% Answering this question is further complicated by the fundamental differences between learning-based and non-learning-based CCs. In particular, CCs differ not only in their internal logic, but also in the signals they consume and the assumptions they make about the network environment, making direct comparison under identical perturbations infeasible. Existing stress-testing approaches~\cite{giacomoni2024reinforcement, mazilu2025learning} often rely on predefined (manually created) evaluation scenarios where identical network conditions are applied to all CCs without adapting to the specific characteristics of each controller. This limits the ability to expose weaknesses that arise only under controller-specific worst-case conditions.
% Other works~\cite{ahmed26advnet,ray2022cc,robust-hotnet19}
% Other works, such as \cut{CC-Fuzz~\cite{ray2022cc}}AdvNet\cite{ahmed2026advnet}, move beyond fixed scenarios by automatically generating network traces that degrade the performance of a given CC.
% \shehab{However, it only modifies environmental conditions and does not consider the effect of input signals on CCs.}
% \cut{However, its focus is on uncovering failure cases for a single CC at a time}
% \shehab{Is your focus different? What I understood is you also generated adversarial cases for one at a time}
% \shehab{Furthermore, it focuses solely on traditional CCs and does not consider \emph{learning-based} approaches.}
% \cut{As a result, it remains unclear how different CCs compare in terms of robustness under adversarially challenging conditions.}
% \shehab{Not sure how the limitations connect to the conclusion you made. We can say ``Moreover, it remains unclear how the discovered adversarial conditions can be leveraged to improve robustness.''}
% \shehab{I couldn't understand what you wanted to convey in this sentence. I think the sentence should clearly state the limitation. You are saying first that it partially addresses the issue, and then you said but many .... So some are fully addressing the issue? Also, what do you mean by shared evaluation scenarios, what is meant by applied uniformly across policies, what is a test condition? Another word came here that is controller, what is a controller?}
% \gang{I think you need a few sentences to highlight how the adversarial agent works.}
% \shehab{Same comment. Too many different phrases here without defining them earlier. network statistics, inputs to the controller, external network conditions.}
\para{Our Approach.}
In this paper, we develop \sysname{}, an adversarial testing framework to systematically evaluate and compare the robustness of learning-based and non-learning-based congestion controllers. \sysname{} introduces an {\em adversarial agent} that operates in a closed loop with the congestion controllers. During the testing, a congestion controller continuously measures/observes network states (e.g., round-trip time, packet loss), and uses them as input signals to make congestion control decisions (e.g., by adjusting the congestion window). Meanwhile, the adversarial agent will also observe the network states and generate bounded perturbations that either (1) modify the direct input signals to the target congestion controller, or (2) change the external network conditions (which will indirectly influence the input signals used by the controller). The congestion controller itself remains unchanged and reacts solely to these modified observations. The adversarial objective is to construct challenging yet constrained conditions that systematically degrade performance while preserving plausibility and interpretability.
% Our approach consists of two complementary levels. First, we perform feature-level perturbation as a diagnostic tool, applying controlled modifications to a shared input signal (minimum RTT) to understand how different CCs respond to corrupted or biased measurements. This stage provides an initial view of signal sensitivity and enables consistent cross-policy comparison, but does not aim to model realistic attacker capabilities. Building on this, we focus on environment-level adversarial testing as the primary evaluation setting. Instead of manipulating internal inputs, we shape external network conditions, such as bandwidth dynamics and delay changes,
% that indirectly influence policy behavior. These conditions are generated under explicit constraints (e.g., bounded smoothness), which serve as tunable factors that allow developers to explore a spectrum of environments ranging from stable to highly challenging. This perspective aligns with practical networking scenarios, where unfavorable dynamics may arise from cross traffic, routing changes, and measurement variability, while also providing a controllable way to study how different constraints impact policy behavior. Using this two-stage evaluation strategy, we conduct an adversarial robustness study of four representative CCs, including two learning-based policies and four widely deployed non-learning-based algorithms, enabling a consistent and policy-aware comparison across heterogeneous designs.
% \gang{do we evaluate delay changes too? if yes, then great}
% \gang{We should name the protocols being considered.}
Our adversarial testing consists of two types of manipulations. First, we perform {\em feature-level} manipulation, applying controlled modifications to a shared input signal to understand how different CCs respond to corrupted or noisy measurements. To enable consistent comparison, in this paper, we focus on an input signal that is widely used by both traditional and learning-based CCs: the minimum Round-Trip Time (RTT). More specifically, we test controller protocols that explicitly rely on RTT-related signals, including two learning-based CCs (Orca~\cite{abbasloo2020classic} and Canopy~\cite{yang2024canopy}) and representative delay-sensitive designs such as TCP Vegas~\cite{brakmo1994tcp} and BBR~\cite{cardwell2017bbr}. This feature-level testing provides an initial view of signal sensitivity and enables cross-policy comparison, but does not aim to model realistic attacker capabilities.
% \shehab{Is this the only feature-level parameter that we perturb? The previous paragraphs gave an impression that we are perturbing multiple of these. I might be wrong.}
% \gang{Separate method and results.}
Second, we consider {\em environment-level} adversarial testing. Instead of directly manipulating the input signals (which can be unrealistic), we shape the external network conditions, such as bandwidth dynamics, which will {\em indirectly} influence the input signals to the congestion controllers. This also allows us to test and compare a broader set of congestion controllers, including two learning-based policies (Orca and Canopy) and four widely deployed non-learning-based controllers (Cubic~\cite{ha2008cubic}, TCP Vegas, TCP LP~\cite{kuzmanovic2003tcp}, and TCP Illinois~\cite{mazilu2025learning}). This covers key CC categories of loss-based, delay-based, and hybrid approaches. The adversarial traces are generated under explicit constraints (e.g., bounded smoothness), which serve as tunable factors that allow CC developers to explore a spectrum of challenging environments.
% This aligns with practical networking scenarios, where unfavorable available bandwidth dynamics may arise from cross traffic, routing changes, and measurement variability, while also providing a controllable way to study how different constraints impact policy behavior.
% \shehab{I don't see the connection between this paragraph and last paragraph. You started with this: Building on this, but I am struggling to find the relation. These two seem to be unrelated.}
% % \gang{Should we make some conclusions on which protocols are more robust, under our experiment settings?}
% \gang{These two paragraphs below are too general and lack evaluation details.
% 1. We need to write down key evaluation results and findings. This is probably the most important thing in the paper. You need to cite key numbers in the evaluation.
% 2. LP case study needs to highlight the insights
% 3. Adversarial testing results should be reported with numbers.
% }
\para{Evaluation and Findings.} Our analysis leads to a series of findings. We briefly highlight the following:
{\em Feature-level perturbation} (\S\ref{sec:feature_level}). We observe that learning-based CCs have a higher level of robustness than non-learning-based CCs under feature-level perturbation. For example, under ±50\% adversarial noise on the input, Canopy degrades by only 3\% in bandwidth utilization, and Orca degrades by 8\%. In comparison, non-learning-based CCs have larger degradation, with BBR dropping by 15\% and TCP Vegas by 10\%.
{\em Environment-level perturbation} (\S\ref{sec:env_level}). Under the more realistic environment-level perturbation, learning-based CCs continue to show higher robustness than non-learning-based CCs. For example, under adversarial bandwidth traces, the degradation of learning-based CCs is modest (12\% for Orca and 7\% for Canopy) but it is more severe for non-learning-based CCs (19\% for Cubic, 17\% for TCP Vegas, 18\% for TCP Illinois, and 30\% for TCP LP).
% Taken together, these results consistently show that learning-based CCs are more robust than non-learning-based designs under both feature-level and environment-level adversarial settings, exhibiting smaller performance degradation across all tested conditions.
{\em Learning-based vs. non-learning-based CCs} (\S\ref{sec:env_reason}, \S\ref{sec:case_study}).
We further analyze the potential reasons behind the different robustness levels of learning and non-learning-based CCs. One hypothesis is that the improved robustness of learning-based CCs may stem from their {\em smoother and more continuous action space}. However, our analysis shows the {\em opposite} direction, as learning-based CCs make more drastic congestion window (cwnd) adjustments than non-learning-based CCs. We further analyze the {\em responsiveness} of cwnd adjustments of these protocols. We show that learning-based CCs are more responsive to bandwidth changes with more precise cwnd adjustments, while non-learning-based CCs rely on rigid control rules that limit their responsiveness. This offers an explanation for their robustness under adversarial bandwidth conditions. Finally, we perform a case study on a non-learning-based CC (TCP LP) that has the most performance degradation. We illustrate how adversarial bandwidth variations consistently trigger TCP LP's conservative design (e.g., early congestion avoidance) to prevent it from using available bandwidth.
% Finally, we perform a case study on a non-learning-based CC (TCP LP) that has the most performance degradation. We illustrate how adversarial bandwidth variations lead to TCP LP overreacting to delay signals and triggering early congestion avoidance before fully exploring available bandwidth.
% \textbf{Understanding robustness through cwnd dynamics.}
% One might expect that the improved robustness of learning-based CCs stems from smoother or more continuous control behavior on cwnd. However, our analysis of cwnd dynamics under adversarial conditions in Section~\S\ref{sec:env_reason} shows the opposite. Learning-based CCs produce significantly more dynamic adjustments, with average absolute slope exceeding 6500–7100, compared to below 1000 for non-learning-based controllers.
% This indicates that their robustness does not arise from smoothness, but rather from a more flexible and expressive control mechanism enabled by a larger action space, allowing finer-grained and more adaptive responses to changing network conditions. In contrast, non-learning-based controllers rely on more rigid control rules and exhibit limited responsiveness, making them more vulnerable to structured adversarial dynamics.
% \textbf{Case study (TCP LP).}
% Beyond aggregate metrics, adversarially generated traces provide actionable insights into controller behavior. For non-learning-based CCs, these traces can serve as diagnostic tools to expose failure modes and understand how specific design choices interact with challenging network dynamics. For example, our case study in Section~\S\ref{sec:case_study} on TCP LP demonstrates how structured bandwidth variations can lead to over-reacting to delay signals, where the protocol triggers early congestion avoidance before fully exploring available bandwidth. This behavior, combined with abrupt reductions in cwnd, results in persistent underutilization, revealing limitations that are difficult to observe under standard evaluation settings.
{\em Adversarial training} (\S\ref{sec:adv_train}). Finally, we show that the adversarial traces generated by \sysname{} can effectively improve the robustness of a learning-based CC. For example, adversarial training increases Orca's bandwidth utilization under adversarial conditions by 6\% (from 85.49\% to 91.66\%) without increasing queuing delay. Also, it maintains a comparable performance on benign workloads (with less than 1\% changes). This illustrates the value of \sysname{} to CC developers for improving their protocols.
% and maintains comparable performance on benign workloads, with less than 1\% change on both random and real-world traces. This highlights the role of the adversarial testing framework not only as an evaluation tool, but also as a practical mechanism for improving congestion control policies.
% In this setting, we observe that learning-based CCs maintain near-stable utilization under both small and large perturbations, with Canopy degrading by only about 3\% and Orca by around 8\% under ±50\% perturbation. In contrast, non-learning-based designs experience significantly larger degradation, with BBR dropping by about 15\% and TCP Vegas by around 10\%.
% Under this more realistic setting, we observe consistent robustness differences: learning-based policies exhibit relatively modest utilization degradation (approximately 12\% for Orca and 7\% for Canopy), whereas non-learning-based policies suffer substantially larger drops, including around 19\% for Cubic, 17\% for TCP Vegas, 18\% for TCP Illinois, and up to 30\% for TCP LP. Taken together, these results consistently show that learning-based CCs are more robust than non-learning-based designs under both feature-level and environment-level adversarial settings, exhibiting smaller performance degradation across all tested conditions.
% One might expect that the improved robustness of learning-based CCs stems from smoother or more continuous control behavior, which could help stabilize performance under adversarial conditions. To further investigate this hypothesis, we analyze the congestion window (cwnd) dynamics of different policies under adversarially generated conditions. Surprisingly, learning-based CCs do not exhibit smoother behavior; instead, they produce significantly more dynamic cwnd adjustments, with the average absolute slope exceeding 6500–7100 compared to below 1000 for non-learning-based designs. This indicates that their robustness does not arise from smoothness, but rather from a more flexible and expressive control mechanism, enabled by a larger action space that allows finer-grained and more adaptive responses to changing network conditions. In contrast, non-learning-based policies rely on more rigid control rules and exhibit limited responsiveness, making them more vulnerable to structured adversarial dynamics that require timely adaptation.
% Beyond quantitative evaluation, our framework also provides actionable insights into CC behavior. For non-learning-based policies, adversarially generated traces can be used as diagnostic tools to expose failure modes and understand how specific design choices interact with challenging network dynamics. For example, our case study on LP demonstrates how structured bandwidth variations can lead to delayed reactions and persistent underutilization, revealing limitations that are difficult to observe under standard evaluation settings~\ref{sec:case_study}.
% For learning-based policies, these adversarial traces can be further leveraged as training signals. By incorporating them into the training process, we enable the policy to learn from challenging conditions and improve its robustness. As shown in our experiments, adversarial training improves utilization under adversarial conditions by over 6\% (from 85.49\% to 91.66\%) while also reducing queuing delay, and maintains comparable performance on benign workloads, with less than 1\% change on both random and real-world traces. This highlights the role of the adversarial testing framework not only as an evaluation tool, but also as a practical mechanism for improving congestion control policies.
% \shehab{I think it will be nice if relevant section is referred when we are talking about a result}
\para{Contributions.}
We make the following contributions:
\begin{itemize}
\item {\em A New Evaluation Framework.}
We introduce \sysname{}, an evaluation framework to compare the robustness of learning and non-learning based CCs. Our experiments show that learning-based CCs exhibit stronger robustness than traditional CCs.
\item {\em A New Network Trace Generation Method.}
We develop an adversarial agent that can generate challenging network conditions while allowing developers to control key characteristics of the adversarial traces (e.g., the level of dynamics, targeted metrics). This enables controlled and reproducible stress testing against CCs.
\item {\em A New Tool for CC Developers.}
We demonstrate how \sysname{} can help developers identify failure cases, understand performance degradation, and improve CC policies through techniques such as adversarial retraining. To support future research and practical use, we will release our code with this paper.
\end{itemize}
% \gang{Mention we will share our code to benefit future research and CC designers.}
\section{Background and Related Work}
\label{sec:related}
Congestion controllers (CCs) are fundamental components of modern network transport protocols, responsible for regulating the sending rate of data flows to efficiently utilize network resources while avoiding congestion collapse. At their core, CCs dynamically adjust control variables such as the congestion window ($cwnd$) or pacing rate based on observed network feedback, such as acknowledgments, delay measurements, and packet loss signals. Over the past decades, a wide range of CCs have been developed, broadly spanning traditional human-designed protocols based on analytical models and heuristics, as well as more recent learning-based approaches that leverage data-driven decision-making.
% In this work, we study adversarial robustness across both learning-based and traditional human-designed congestion control algorithms. Traditional, non-learning-based CCs are built on explicit control logic and are generally interpretable, making their behavior easier to analyze and reason about. In contrast, learning-based CCs have demonstrated strong empirical performance and adaptability, but they often operate as black boxes, making it difficult to predict how they would behave under unseen conditions. Prior studies~\cite{abbasloo2020classic, yang2024canopy} have shown that such learning-based policies can exhibit unexpected behaviors or sudden performance degradation when encountering inputs that deviate from their training distribution. These fundamental differences motivate a unified evaluation framework that can systematically assess robustness across both paradigms, enabling a deeper understanding of their failure modes and the triggering conditions.
% \gang{The above two paragraphs are nicely written. Could shorten them if space is needed.}
\subsection{Non-learning-based Congestion Controllers}
\label{sec:related-nonlearning-based}
Traditional CCs are typically manually designed, based on analytical models and heuristic rules.
% that map the observed network signals to control actions.
% These approaches typically adjust the congestion window or pacing rate based on feedback such as packet loss, delay, or estimated bandwidth. Over decades of development, a rich design space of non-learning-based CCs has emerged, which can be broadly categorized based on their primary congestion signals and control strategies.
\para{Loss-based Algorithms.}
Loss-based CCs interpret {\em packet loss} as the primary indicator of congestion. They introduce different variants of the additive-increase multiplicative-decrease (AIMD) paradigm. TCP Reno represents the canonical design. Improvements over Reno include high-speed adaptations such as BIC~\cite{xu2004binary}, Cubic~\cite{ha2008cubic}, HighSpeed TCP~\cite{floyd2003highspeed}, H-TCP~\cite{leith2004h}, Hybla~\cite{caini2004tcp}, and Scalable TCP~\cite{kelly2003scalable}, which modify the window growth function to better utilize high bandwidth-delay product (BDP) networks. These algorithms are widely deployed due to their simplicity and robustness, but their reliance on packet loss as a congestion signal often leads to persistent queue buildup and increased latency.
\para{Delay-based Algorithms.}
Delay-based CCs infer congestion from variations in round-trip time (RTT), aiming to detect queue buildup before packet loss occurs. TCP Vegas~\cite{brakmo1994tcp} is a representative design that adjusts its sending rate based on deviations between expected and observed throughput. Subsequent variants such as TCP Veno~\cite{fu2003tcp} and CDG~\cite{hayes2011revisiting} extend this idea by incorporating additional heuristics to improve robustness under noisy conditions. While delay-based approaches can achieve low latency and stable behavior, they are sensitive to measurement noise and may under-utilize available bandwidth.
\para{Hybrid Algorithms.}
Hybrid CCs combine multiple congestion signals, typically integrating both loss and delay information to balance utilization and latency. TCP Illinois~\cite{liu2006tcp} and TCP YeAH~\cite{baiocchi2007yeah} adapt their window adjustment rules based on both delay trends and packet loss events, while TCP Westwood~\cite{mascolo2001tcp} incorporates bandwidth estimation to refine its response after congestion. TCP LP~\cite{kuzmanovic2003tcp} further extends this design space by leveraging one-way delay measurements to detect early congestion and yield bandwidth to competing flows. These hybrid approaches aim to improve adaptability across diverse network conditions, but their performance depends on the interaction between multiple control signals and parameters.
\para{Explicit-state Estimation Designs.}
Instead of directly reacting to the observed signals, some CCs estimate underlying network state variables, such as bottleneck bandwidth, propagation delay, or queue occupancy, and derive control actions from these estimates. For example, TCP BBR~\cite{cardwell2017bbr} continuously estimates the bottleneck bandwidth and minimum round-trip time (RTT), and sets its sending rate based on the inferred bandwidth-delay product (BDP), rather than directly responding to packet loss or delay increase. Similarly, DCTCP~\cite{alizadeh2010data} uses explicit congestion notification (ECN) signals to approximate the level of queue occupancy and adjusts its congestion window proportionally to the extent of congestion.
% Overall, non-learning-based CCs span a wide range of design philosophies, from loss-driven heuristics to delay-aware and model-based approaches. Their explicit control logic makes them interpretable and easier to analyze, but also exposes them to potential weaknesses when underlying assumptions are violated or when input signals are systematically perturbed. Unlike learning-based approaches, these designs rely on simple estimation rules and analytical relationships rather than data-driven training. Their effectiveness therefore depends on the accuracy of the estimated network state, which may degrade under dynamic or adversarial conditions.
% \subsection{learning-based Congestion Control Policies}
% Recent advances in machine learning have led to a new class of congestion control algorithms that learn control policies directly from data rather than relying on handcrafted rules. These approaches typically formulate congestion control as a sequential decision-making problem, where a policy maps observed network statistics to control actions.
% Recent work has explored the use of deep reinforcement learning (RL) to learn congestion control policies. For example, Aurora~\cite{jay2019deep} formulates congestion control as an RL problem, where a neural policy maps a history of network observations (e.g., latency, throughput, and loss) to rate adjustments. In parallel, the PCC Vivace~\cite{dong2018pcc} framework adopts an online learning method, where the sender continuously evaluates the utility of its actions and adjusts its sending rate accordingly. It is further improved by incorporating gradient-based optimization to achieve faster convergence and improved stability. \zhi{Glider~\cite{xia2023glider} proposes a fully learning-based congestion control framework that applies deep reinforcement learning to directly learn congestion window adjustment policies from network feedback. To enable RL in continuous network environments, it introduces a discretization mechanism that structures packet transmission into decision steps, allowing the agent to optimize throughput, delay, and loss jointly.
% Complementary to fully learning-based approaches, hybrid designs combine machine learning with rule-based control, where learning may support state estimation or directly influence control decisions. MLACC~\cite{elbery2023toward} adopts a hybrid design that uses a supervised learning model to estimate the bottleneck bandwidth utilization from observed network statistics, while relying on a heuristic control policy to adjust the sending rate. Unlike fully learning-based approaches, this design decouples state estimation from control, improving interpretability and stability while still benefiting from learned network signals.} Orca~\cite{abbasloo2020classic} introduces a hybrid design that combines a traditional TCP backbone with a deep RL agent that periodically adjusts the congestion window, enabling improved adaptability while retaining the stability of classical control logic. However, researchers find that such learning-based policies can exhibit sensitivity to noise and may enter persistent low-performance states under unforeseen inputs. To address these limitations, Canopy~\cite{yang2024canopy} integrates {\em formal verification} into the learning loop, providing robustness and performance certificates that guide training toward improved worst-case behavior.
% Finally, researchers have also explored learning from existing congestion control algorithms. Sage~\cite{yen2023computers} proposes an offline reinforcement learning framework that learns a new congestion control policy by leveraging a dataset collected from a diverse pool of existing non-learning-based CCs, aiming to surpass individual handcrafted schemes without requiring online interaction during training. This line of work highlights an alternative paradigm where learning is guided by historical policy behavior rather than direct environment interaction.
\subsection{Learning-based Congestion Controllers}
Recent advances in machine learning have led to a new class of congestion controllers that learn control policies directly from data rather than relying on handcrafted rules. These approaches typically formulate congestion control as a sequential decision-making problem, where a policy maps observed network statistics to control actions.
Early work has explored the use of reinforcement learning (RL) to learn congestion control policies. For example, Aurora~\cite{jay2019deep} formulates congestion control as an RL problem, where a neural policy maps a history of network observations (e.g., latency, throughput, and loss) to rate adjustments. PCC Vivace~\cite{dong2018pcc} adopts an online learning approach that continuously evaluates the utility of actions and updates the sending rate via gradient-based optimization. Glider~\cite{xia2023glider} applies deep reinforcement learning to directly learn congestion window adjustment policies by discretizing continuous network interactions into decision steps.
Recently, hybrid designs combine machine learning with rule-based control, where learning may support state estimation or directly influence control decisions. MLACC~\cite{elbery2023toward} uses a supervised model to estimate bottleneck utilization while relying on heuristic control for rate adjustment. Orca~\cite{abbasloo2020classic} integrates a deep RL agent into a TCP framework to periodically adjust the congestion window. Building on this line of work, Canopy~\cite{yang2024canopy} addresses the sensitivity and instability of learning-based policies under unforeseen inputs by incorporating {\em formal verification} into the learning loop, providing robustness and performance guarantees.
Finally, researchers have also explored learning from existing congestion control algorithms. Sage~\cite{yen2023computers} proposes an offline reinforcement learning framework that learns a new policy from datasets collected from diverse non-learning-based CCs.
% highlighting an alternative paradigm where learning is guided by historical policy behavior rather than direct environment interaction.
\subsection{Benchmarks for Congestion Controllers}
Researchers have examined the performance of CCs but prior works have largely focused on {\em average-case} behavior under fixed or benign conditions, leaving {\em worst-case} robustness insufficiently explored.
Giacomoni et al.~\cite{giacomoni2024reinforcement} present one of the first large-scale evaluations of reinforcement learning (RL)-based congestion controllers. Their results show that RL-based approaches can quickly utilize available bandwidth but often suffer from unfairness and degraded performance under dynamic conditions. Mazilu et al.~\cite{mazilu2025learning} extend this line of work by conducting an in-depth comparison between learning-based and human-designed CCs, highlighting generalization and instability issues when operating outside training regimes. However, both studies rely on predefined scenarios, fixed traces, or manually constructed benchmarks to evaluate controller behavior: they do not systematically explore the space of challenging network conditions or worst-case scenarios.
To move beyond manually constructed scenarios, researchers propose to automatically generate network traces that degrade CC performance~\cite{ahmed26advnet,ray2022cc,robust-hotnet19}. By evolving traces using performance-driven fitness functions, they adapt the generated scenarios to the specific congestion controller to discover unexpected (and often non-obvious) behaviors.
Compared with existing work, our paper has {\em three novel aspects}. First, existing works~\cite{ahmed26advnet,ray2022cc,robust-hotnet19} still focus on traditional CCs. They did not focus on comparing learning-based CCS with non-learning-based CCs. In addition, they mainly focus on modifying network environment conditions without evaluating the impact of input signals of CCs (i.e., feature-level perturbations). Finally, they did not use adversarial training to improve learning-based CCs.
% \cut{Also, CC-Fuzz's evaluation is conducted using NS3 simulation (instead of emulating the real protocol implementations)}.
% \shehab{This one is true for CC-Fuzz only. Can we say ``Also, they do not utilize the discovered challenging conditions to improve robustness of the CC. Same as we stated in intro?}
% To move beyond manually constructed scenarios, CC-Fuzz~\cite{ray2022cc} and AdvNet~\cite{ahmed2026advnet} automatically generates network traces that degrade CC performance. By evolving traces using performance-driven fitness functions, \cut{CC-Fuzz adapts} \shehab{they adapt} the generated scenarios to the specific congestion controller to discover unexpected (and often non-obvious) behaviors. However, their primary focus is on anomalous behaviors of individual CCs, rather than enabling systematic, cross-policy comparison. More importantly, they target traditional CCs without considering {\em learning-based approaches} such as Orca and Canopy.
% \cut{Also, CC-Fuzz's evaluation is conducted using NS3 simulation (instead of emulating the real protocol implementations)}.
% \shehab{This one is true for CC-Fuzz only. Can we say ``Also, they do not utilize the discovered challenging conditions to improve robustness of the CC. Same as we stated in intro?}
% Other works~\cite{ahmed26advnet,ray2022cc,robust-hotnet19} move beyond fixed scenarios by automatically generating network traces that degrade the performance of a given CC.
% However, these works still focus on traditional, non-learning-based CCs without considering/comparing against {\em learning-based CCs}. Also, they all focus on modifying the network environmental conditions without evaluating the impact of the input signals to CCs.
% Overall, prior work has advanced the evaluation of CCs along two complementary directions. Benchmarking studies focus on performance under fixed traces or controlled variability, capturing typical operating behavior, while search-based approaches such as CC-Fuzz demonstrate the ability to uncover corner cases through automated exploration of the environment space. However, these approaches leave an important gap. Benchmarking methodologies do not explicitly target worst-case behavior, while search-based techniques often produce scenarios that are difficult to interpret, lack structured constraints, and do not support consistent comparison across different controllers. As a result, there is limited understanding of how CCs behave under systematically constructed worst-case conditions that remain plausible, comparable, and informative for analysis and design.
% This gap motivates us to develop an adversarial testing framework that bridges these perspectives by combining systematic worst-case exploration with controlled and interpretable evaluation, enabling both consistent cross-policy comparison and practical tools for diagnosing and improving congestion control designs.
% \insight{Existing evaluations either focus on average-case performance or generate unconstrained worst-case scenarios, leaving a gap in understanding robustness under structured and realistic adversarial conditions.}
\section{Adversarial Testing Framework}
\label{sec:method}
\begin{figure}[t]
\centering
\begin{minipage}[t]{0.5\textwidth}
\includegraphics[width=0.99\textwidth]{ldos_framework-3.pdf}
\end{minipage}
\caption{
\sysname: our adversarial testing framework for the congestion controller (CC). The CC can be learning-based or non-learning-based. ``cwnd'' means ``congestion window.''
}
\label{fig:framework}
\vspace{-0.12in}
\end{figure}
% \gang{There are different ways to construct the adv testing framework, why we use RL-based? We need some reasoning: easier to design constraints into rewards, dynamically adapting to changes in real-time? I don't know which arguments you had in mind, but we need to say something.
% }
% \zhi{maybe a figure to show the overall framework}
% There are multiple possible approaches for constructing an adversarial testing framework, including random search, heuristic perturbation, or evolutionary methods such as genetic algorithms. We adopt reinforcement learning (RL) as the adversarial engine due to its natural fit for congestion control.
In this section, we provide an overview for \sysname. We use reinforcement learning (RL) as the adversarial engine to construct our testing framework for a few considerations. First, congestion controllers (CCs) naturally formulate an RL-like interaction loop with the network, where network signals influence CC's actions, and the actions will further influence the network conditions in the future. An ``adversarial'' RL agent can be easily plugged into this loop to capture and learn from such dynamic interactions. Second, using RL allows us to explicitly encode adversarial objectives and constraints through reward design and bounded action spaces. This allows us to create ``targeted'' adversarial traces to stress test CCs to expose failure modes and perform diagnosis. Ultimately, we want to provide testing tools for CC protocol designers to better understand the weaknesses of their designs.
\para{Framework Overview.}
Fig.~\ref{fig:framework} illustrates the high-level architecture of our testing framework, \sysname{}.
% making it easier to control realism and strength compared to heuristic or search-based methods. Finally, since modern learning-based CCs (e.g., Orca and Canopy) are themselves RL-based, modeling the adversary as an RL agent provides a unified and intuitive framework.
% Our adversarial testing framework (Figure~\ref{fig:framework}) builds upon a standard learning-based congestion control architecture, as exemplified by Orca~\cite{abbasloo2020classic} and adopted by recent designs such as Canopy~\cite{yang2024canopy}.
% \shehab{give at least an example}
% \shehab{Use any one. Either statistics or states. I guess state is more appropriate}
Congestion control is formulated as a sequential decision-making problem. A congestion controller (CC) in the green box will collect network states (e.g., min RTT) from the kernel at a fixed monitoring interval. The network states are passed to the user-space, where the CC produces control actions (e.g., adjustments to the cwnd). The actions are then enforced at the transport stack while the underlying TCP logic continues its fine-grained ACK-driven updates. Here, the CC can be either a learning-based protocol or a non-learning-based protocol.
% \shehab{For state update right? For environment update, is it also true?}
As shown in Fig.~\ref{fig:framework}, we introduce an adversarial agent (red box) into this observation-control pipeline. The adversarial agent seeks to manipulate the states observed by the target CC to influence its actions. The congestion controller itself remains unchanged and reacts solely to the perturbed input network states (i.e., input signals) it receives. In this paper, we consider two ways to manipulate the input network states: (1) feature-level manipulation, which directly perturbs the input signals used by the CC (e.g., RTT measurements), and (2) environment-level manipulation, which modifies underlying network conditions (e.g., bandwidth) to indirectly influence these observations. From a systems perspective, the adversarial agent operates externally to the congestion controller, without modifying protocol internals such as congestion window updates, pacing mechanisms, or state transitions. This design preserves the original behavior of each congestion controller while enabling systematic and controlled adversarial testing.
% \shehab{I believe we should have a complete but possibly abridged description of full diagram here. The description of reward is missing here. Also, what does the arrow from CC to environment mean? What is cwnd doing to environment? From the figure, it is not clear that adversarial agent is manipulating environment and state. The fact that state and reward both are on one arrow is confusing. What does the arrow from environment to CC mean? What's its relation to state?
% My suggestion: State will be another box. CC takes info both from environment and state. You can mention in state box that it is kernel space. Move the feature-level manipulation and environment-level manipulation a bit more towards the arrow to more clearly distinguish between the two. Reward should be part of adversarial agent, right? How do you compute it? If it is from states, then probably another arrow to state from reward box.
% }
% \zhi[change the network states and thus CC's observations. This sentence is still confusing of network states. In the feature-level, we also have network state there. ]
% \zhi{(1) feature-level manipulation, which directly perturbs the observed signals used by the CC (e.g., RTT measurements), and (2) environment-level manipulation, which modifies underlying network conditions (e.g., bandwidth and link delay) to \emph{indirectly} influence these observations.}
% \para{Threat Model.}
% where a learning agent operates at a coarse-grained timescale alongside an underlying transport protocol. Network statistics are collected from the kernel at fixed monitoring intervals and passed to a user-space module as observations. Based on these observations, the agent produces control actions (e.g., adjustments to the congestion window), which are then enforced back into the transport stack while the underlying TCP logic continues its fine-grained ACK-driven updates. This design enables continuous interaction between the network environment and the learning agent, allowing policies to adapt dynamically to changing network conditions. In practice, both Orca and prior work rely on the Mahimahi emulator~\cite{netravali2015mahimahi} to provide controlled yet realistic network environments, where real packet transmissions over emulated links enable faithful evaluation of congestion control behavior.
% We extend this learning-based architecture by introducing an adversarial agent into the observation and control pipeline. Instead of directly feeding monitored signals to the congestion control policy, the adversarial agent first observes the same network statistics and generates a modified version under explicitly defined constraints. The congestion control policy itself remains unchanged and reacts solely to the (potentially perturbed) inputs it receives. From a systems perspective, the adversarial agent operates at the interface between monitoring and control, without modifying protocol internals such as congestion window updates, pacing mechanisms, or state transitions. This design preserves the original behavior of each congestion control algorithm while enabling systematic and controlled perturbation of its inputs or environment.
\para{Threat Model.}
% \gang{I think we need a figure here to show how the adversarial agent works. You can also use this figure to explain what feature-level and environment-level attacks work too. Using draw.io to draw one and save the PDF and source files to overleaf.}
% Importantly, the adversarial framework does not assume a fixed control surface. Depending on the evaluation objective, the adversarial agent can operate on different components, such as input features or environment-level variables. This flexibility allows the same framework to support both feature-level perturbation (as a diagnostic tool) and environment-level manipulation (as a more realistic evaluation setting).
% \shehab{what does shared mean, probably I had a similar comment in the intro}
\sysname{} explicitly supports two complementary control surfaces as shown in Fig.~\ref{fig:framework}. In (1), the adversarial agent performs \emph{feature-level manipulation} by directly perturbing input signals (e.g., min RTT) before they are consumed by the congestion controllers. This provides a controlled diagnostic setting to probe how sensitive different CCs are to corrupted or noisy measurements. In (2), the agent performs \emph{environment-level manipulation} by modifying environment variables, such as bandwidth or link delay, thereby indirectly influencing the observations for the CCs.
% \shehab{in intro you used levels, none seem the best choice honestly. I feel we can use ``methods'' here.}
% \shehab{I think you said it earlier too. I didn't understand this.}
These two types of adversarial testing serve complementary purposes. Feature-level manipulation isolates the dependence of CCs on the specific input signals and enables cross-policy comparison. In contrast, environment-level manipulation reflects more {\em realistic conditions}, where performance degradation arises from adverse network dynamics rather than direct input corruption. We will provide more detailed justifications for these two adversarial testing types (Sections~\ref{sec:feature_level} and \ref{sec:env_level}).
\para{Agent Training.}
The adversarial agent is trained online using reinforcement learning. At each monitoring interval, it observes network states and produces actions that modify a designated control surface, with the resulting system behavior determining the adversarial RL reward.
To regulate adversarial strength, all actions are subject to explicit constraints that bound their magnitude, temporal smoothness, or long-term variability. These constraints act as tunable parameters, enabling systematic exploration of different adversarial regimes. By adjusting them, CC developers can examine how policies respond under varying degrees of challenge, from mild perturbations to highly structured adversarial conditions. Specific control surfaces and constraint configurations are described in the corresponding experimental sections.
% \subsection{Evaluation Targets and Metrics}
% \zhi{introduce random baseline? Is the random baseline not needed for the feature space? Random baseline in environment manipulation is more like a "noise" perturbation in feature manipulation.}
% We evaluate six representative congestion control algorithms, including two learning-based policies (Orca~\cite{abbasloo2020classic} and Canopy~\cite{yang2024canopy}) and four widely deployed non-learned designs (Cubic~\cite{ha2008cubic}, TCP Vegas~\cite{brakmo1994tcp}, TCP LP~\cite{kuzmanovic2003tcp}, and TCP Illinois~\cite{liu2006tcp}).
% These non-learned CCs span the major design categories discussed in Section~\ref{sec:related-nonlearned}, including loss-based (Cubic), delay-based (TCP Vegas), and hybrid approaches (TCP LP and TCP Illinois). These algorithms differ in their control logic, signal dependencies, and design philosophy, providing a diverse testbed for robustness evaluation. Our goal is not exhaustive coverage of all CCs, but a principled comparison between learned and non-learned approaches under a unified adversarial testing framework. All policies are evaluated without modification to their internal logic. The adversarial framework interacts with each policy only through exposed input signals or externally observable network conditions, ensuring that the evaluation reflects behavioral robustness rather than implementation-specific artifacts.
\para{Implementation and Evaluation Metrics.}
Instead of using network simulators (e.g., NS3), we implement the framework on a network emulator Mahimahi~\cite{netravali2015mahimahi} to experiment with more realistic network environments and test real CC implementations. This emulator is used by many existing learning-based CCs~\cite{abbasloo2020classic, yang2024canopy,ahmed2026advnet} as it supports real packet transmissions over emulated links, enabling faithful evaluation of congestion control behavior. We use the network traces from~\cite{yang2024canopy}, which capture a wide range of dynamic network conditions.
% \gang{Did we also perform the evaluation on real-world traces too? The security community may be a bit suspicious about synthetic traces.}
Following the standard setup in prior work~\cite{yang2024canopy}, each trace is emulated using Mahimahi with a fixed link delay of 10\,ms and a buffer size set to $2\times$BDP, representing a typical operating regime. Unless otherwise specified, this configuration is used consistently across all experiments.
% \gang{be more explicit, how many runs?}
We measure performance using two primary metrics: {\em bandwidth utilization} and {\em queuing delay}. Bandwidth utilization captures how effectively a CC utilizes available bandwidth (higher is better), while queuing delay reflects its responsiveness and stability under congestion (lower is better). Together, these metrics characterize the key trade-offs in congestion control performance. All results are reported after adversarial training has converged and are averaged over three runs to account for stochasticity in both network dynamics and adversarial learning.
% \shehab{I think this section should be re-organized. Suggestions:
% 1. There is no subsections here. All important submodule should have its own subsection.
% 2. In the description of thread model, you need to start introducing notations to describe things. For instance, how do you manipulate in the feature-level or environment-level? What exactly is modified and how? Bring equations/formula and talk about the granularity of adversarial manipulation. Was there any system challenge there? Any tradeoff? Any design decision you had to take?
% 3. In agent training, you need to explicitly define what are the states, action and reward. You talked about regulation but from a high level, again talk about the budget concept, bring equation of slope and talk about tradeoff of high/low budget. Reward calculation should also be here. We need an equation for the reward.
% 4. The last para seems part of experiment, should be moved out of this section.
% 5. There is no section for adversarial training. It should contain equation(s). an optional figure and detailed description of the method.
% }
\section{Feature-level Adversarial Testing}
\label{sec:feature_level}
We begin our evaluation with a feature-level sensitivity analysis. The goal of this stage is not necessarily about modeling realistic adversarial capabilities. Instead, we seek to obtain an initial understanding of how congestion control algorithms respond to corrupted, biased, or extreme input signals. It allows us to directly probe the dependence of different policies on specific signals. From a protocol design perspective, this approach can serve as a stress-testing tool by injecting arbitrary feature values, enabling designers to systematically evaluate robustness under extreme or unexpected input conditions. We acknowledge that this setting may not be ideal to model attackers (e.g., it makes strong assumptions on attacker capabilities, such as compromising the measurement components or directly interfering with internal signal processing). Instead, we use this setting to capture undesired scenarios such as measurement noise, estimation bias, or transient inaccuracies in signal computation. The analysis serves as a diagnostic tool to measure feature sensitivity and identify signals that are critical to policy behavior.
% These insights motivate and inform the more realistic environment-level adversarial experiments presented in the next section.
% \gang{I like the arguments you maded for the feature level.}
% \subsection{Feature-level Control Surface}
% A key challenge in comparing learning- and non-learning-based congestion control policies is that they rely on different sets of input signals. learning-based policies typically consume a high-dimensional feature vector derived from network measurements, while non-learning-based policies are driven by hand-crafted control logic based on a smaller set of signals. As a result, direct and uniform manipulation of all policy inputs is infeasible.
% To enable cross-policy comparison, we focus on input signals that are either explicitly or implicitly relied upon by all evaluated CCs. In particular, we identify the minimum RTT estimate as a shared and influential signal across both learning- and non-learning-based designs. Minimum RTT plays a central role in delay estimation, pacing decisions, and congestion window adjustment, making it a natural candidate for feature-level sensitivity analysis.
\subsection{Feature-level Control Surface}
As discussed in Section~\ref{sec:related}, learning and non-learning-based CCs rely on different sets of input signals (or features). This is especially true for traditional, human-designed CCs, as they often rely on one or a few distinct input signals to make decisions (e.g., loss-based algorithms mainly monitor package loss while completely ignoring delay features such as RTT). As such, it is difficult to manipulate certain features while fairly comparing different algorithms. To enable cross-policy comparison, our idea is to identify {\em common input signals} explicitly or implicitly used by most learning and non-learning based CCs and explore how they react to the manipulations on these features. In this paper, we focus on minimum RTT (or min-RTT) as it plays an important role in delay estimation, pacing decisions, and congestion window adjustment, and is widely used in delay-based and hybrid congestion control algorithms. Also, it is a key input signal considered by all learning-based CCs. We understand that min-RTT does not universally apply to all CCs, particularly those that only rely on loss-based signals. In this paper, we use min-RTT to demonstrate our adversarial agent design. The same methodology can be directly applied to other input signals to design adversarial agents (pacing rate, acknowledgment statistics) to measure their influence on CCs that depend on them. This flexibility allows protocol developers to adapt the framework to different subsets of congestion control algorithms based on their specific design characteristics.
% \nic{might also be a good reason in addition to its widely used by CCs: min RTT can be trivial to tamper for certain adversaries.}
% \gang{Does it cover all protocols? How many protocols does it cover based on your surgery? Maybe we could make an argument that min RRT allows us to evaluate more protocols in this paper, but developers can easily adapt this to any feature X to compare two protocols that both depend on feature X}
% A key challenge in comparing learning- and non-learning-based congestion control policies is that they rely on different sets of input signals. Learning-based policies typically consume a high-dimensional feature vector derived from network measurements, while non-learning-based policies are driven by hand-crafted control logic based on a smaller set of signals. As a result, direct and uniform manipulation of all policy inputs is infeasible.
% \nic{feels the logic of the intro paragraph is inverted: goal of the section has been adv. testing and understand each CC's policies, but the need of comparison suddenly comes from nowhere. Seems the natural logic should be: LCC and traditional CCs have drastically different set of signals making it infeasible for us to customly manipulate all policy inputs of each. Thus, it's a key challenge for us to select (a set of) input signal(s) to enable cross-policy comparison. Or, the need of establish cross-CC comparison should be explained earlier in Sec. IV.}
% To enable cross-policy comparison, we focus on input signals that are either explicitly or implicitly relied upon by a broad class of congestion control algorithms. In this work, we select the minimum RTT estimate as a shared and influential signal across both learning- and non-learning-based designs. Minimum RTT plays a central role in delay estimation, pacing decisions, and congestion window adjustment, and is widely used in delay-based and hybrid congestion control algorithms, as well as in model-based designs such as BBR. This makes it a natural candidate for feature-level sensitivity analysis, allowing us to cover a diverse subset of protocols under a common control surface.
% We note that minimum RTT does not universally apply to all congestion control algorithms, particularly those that rely purely on loss-based signals. However, our goal is not to exhaustively cover all possible designs, but to provide a principled and consistent comparison across a representative set of policies that share common signal dependencies. More generally, the choice of minimum RTT reflects one instantiation of our framework rather than a fundamental limitation. The same methodology can be readily extended to other shared signals, such as pacing rate, acknowledgment statistics, or delay-related features, enabling targeted comparison between protocols that depend on a given input signal. This flexibility allows protocol developers to adapt the framework to different subsets of congestion control algorithms based on their specific design characteristics. \nic{might also be a good reason in addition to its widely used by CCs: min RTT can be trivial to tamper for certain adversaries.}
\subsection{Adversarial Agent Design}
\label{sec:feature-design}
CCs are optimizing their decisions to achieve high bandwidth utilization while reducing queuing delay (and avoiding network congestion). Most learning-based CCs design their reward function to achieve these goals. For instance, Orca and Canopy use the following reward function to jointly capture throughput efficiency, packet loss penalty, and delay sensitivity. The reward at time step $t$ is defined as:
\begin{equation}
R_t = \frac{T_t - \lambda L_t}{B_{\max}} \cdot D_t
\end{equation}
where $T_t$ denotes the achieved throughput, $L_t$ is the packet loss rate, $\lambda$ is the loss penalty coefficient, and $B_{\max}$ represents the maximum available bandwidth used for normalization.
The delay-related factor $D_t$ is defined as:
\begin{equation}
D_t =
\begin{cases}
\frac{\gamma \cdot RTT_{\min}}{RTT_t}, & \text{if } \gamma \cdot RTT_{\min} < RTT_t \\
1, & \text{otherwise}
\end{cases}
\end{equation}
where $RTT_t$ is the smoothed round-trip time, $RTT_{\min}$ is the minimum observed RTT, and $\gamma$ is a delay margin coefficient.
\para{An Naive Adversarial Agent Design.}
The adversarial objective is to degrade the utilization--delay trade-off achieved by the congestion controller. With this consideration, a {\em naive} design is to simply define the adversarial reward as {\em the negative} of the original reward used to train learning-based controllers:
\begin{equation}
R^{\text{adv}}_t = - R_t
\end{equation}
thereby encouraging behaviors that move the system away from its intended operating point.
% Specifically, we focus on noise in the estimation of the minimum RTT and allow the adversarial agent to manipulate this signal within a constrained range around its true value.
In terms of capability, we model the adversary as performing bounded perturbations on input signals to reflect uncertainty in network measurements.
Specifically, we perturb the estimated minimum RTT within a constrained range around its true value. We consider both small perturbations (e.g., around 5\%), which capture realistic measurement noise and estimation bias, and larger perturbations (up to 50\%), which are used to stress-test controller behavior under stronger adversarial influence. The latter does not aim to model typical noise levels, but rather to explore the limits of policy robustness when the adversarial capability is significantly amplified.
% Formally, the perturbed minimum RTT is defined within an interval of the form $\text{minRTT} \cdot [1 - x\%, 1 + x\%]$, where $x$ controls the perturbation magnitude.
Formally, the perturbed minimum RTT lies within $\text{minRTT} \cdot [1 - x\%, 1 + x\%]$, where $x$ controls the perturbation magnitude.
\begin{figure}[t]
\centering
\begin{minipage}[t]{0.4\textwidth}
\includegraphics[width=0.99\textwidth]{feature_minrtt_05_all_policies.pdf}
\end{minipage}
\caption{
Feature-level adversarial testing results with 50\% adversarial capacity, under the naive reward design.
}
\label{fig:feature05}
\vspace{-0.12in}
\end{figure}
We implemented this adversarial agent and performed preliminary experiments. The results highlight a {\em key limitation} of this naive approach. More specifically, the adversarial agent tries to simultaneously attack bandwidth utilization and queuing delay (e.g., reducing utilization and increasing delay). At the same time, bandwidth utilization and queuing delay have an inherent trade-off between themselves (e.g., lower bandwidth utilization often reduces queuing delay). As a result, the agent may attack different CCs along different dimensions, making it difficult to compare and interpret the degradation across CCs. Figure~\ref{fig:feature05} illustrates this problem. We show that the agent mainly degrades the performance of Canopy and Orca by increasing their queuing delay. However, for BBR and Vegas, the agent managed to significantly degrade their bandwidth utilization, but with reduced queuing delay as well. Due to the inherent trade-off between bandwidth utilization and queuing delay, it is unclear whether the degradation of bandwidth utilization reflects a true performance degradation from the adversarial agent or a natural trade-off from the reduced queuing delay. It is also difficult to compare the performance of BBR and Vegas with the other protocols.
\para{The Improved Design of Adversarial Agent.}
% A key challenge in evaluating the adversarial agent is to interpret the trade-off between utilization and queuing delay. In many cases, these two metrics move in opposite directions: higher utilization is often achieved by sending more traffic to fully utilize the available bandwidth, which in turn increases queue occupancy and leads to higher queuing delay. As a result, it is difficult to determine whether an observed change reflects a meaningful performance degradation or simply a natural trade-off inherent to congestion control. This ambiguity is further amplified in adversarial settings, where it is unclear whether performance changes are caused by adversarial manipulation or by normal system dynamics.
% \gang{Does this problem also apply to the feature-level manipulation? If so, why not design this for feature-level too? This could be a potential reviewer question.}
To address this problem, we want to have more explicit controls on the adversarial agent's behaviors.
Instead of jointly optimizing multiple competing metrics, we primarily focus on utilization as the optimization objective, while treating queuing delay as a constraint. Specifically, we require that the queuing delay under adversarial conditions {\em should be no lower than} the original/baseline performance while the agent optimizes to degrade the bandwidth utilization. This is to ensure that any degradation in bandwidth utilization is not achieved by trivially reducing congestion.
Here, we take {\em bandwidth utilization} as the main attack target and make {\em queuing delay} a constraint. This mainly considers their practical impact on service quality in real-world scenarios. In other words, bandwidth is a more important factor and a more realistic adversary's attack goal. Today, most CCs for TCP are optimized for bulk reliable data transfer applications in fast, long-distance networks where bandwidth utilization is a higher-prioritized goal. In addition, RTT usually overwhelms queuing delay by magnitude, and most TCP applications are not latency-sensitive. As such, taking queuing delay as the primary attack target is not well justified.
% While queuing delay is an important factor that CCs use for congestion estimation, it is not a factor controlled by CCs.
% We believe in the Linux context queuing delay is more dependent on the queuing disciplines, not on CCAs, and CCAs only reactively behave upon queuing delay measurements. Thus, an adversarial attack aiming to increase queuing delay, while possible to degrade TCP performance ultimately, it might be targeting kernel components outside our scope of CCA robustness.
% In other words, bandwidth is a more important factor and a more realistic adversary's attack goal. We believe so because most CCAs for TCP are optimized for bulk reliable data transfer applications in fast, long-distance networks, where not only bandwidth utilization is a higher prioritized goal, but also RTT usually overwhelms queuing delay by magnitude in such applications and most TCP applications are not that latency-aware so that an attack to queuing delay has adverse effect. On the other hand, while queuing delay is an important factor that CCAs utilize for congestion estimation, it is not a factor single-handed controlled by CCAs. We believe in the Linux context queuing delay is more dependent on the queuing disciplines, not on CCAs, and CCAs only reactively behave upon queuing delay measurements. Thus, an adversarial attack aiming to increase queuing delay, while possible to degrade TCP performance ultimately, it might be targeting kernel components outside our scope of CCA robustness.}
% Here, we take bandwidth utilization as the main attack target and make queuing delay a constraint, mainly considering their practical impact on service quality in real-world scenarios. \nic{In other words, bandwidth is a more important factor and a more realistic adversary's attack goal. We believe so because most CCAs for TCP are optimized for bulk reliable data transfer applications in fast, long-distance networks, where not only bandwidth utilization is a higher prioritized goal, but also RTT usually overwhelms queuing delay by magnitude in such applications and most TCP applications are not that latency-aware so that an attack to queuing delay has adverse effect. On the other hand, while queuing delay is an important factor that CCAs utilize for congestion estimation, it is not a factor single-handed controlled by CCAs. We believe in the Linux context queuing delay is more dependent on the queuing disciplines, not on CCAs, and CCAs only reactively behave upon queuing delay measurements. Thus, an adversarial attack aiming to increase queuing delay, while possible to degrade TCP performance ultimately, it might be targeting kernel components outside our scope of CCA robustness.}
% \gang{Nicholas, can you add your arguments here with a cohesive paragraph, using 2-3 most important reasons?}
% \gang{Why treat the delay as a constraint to maximize the utility degradation? Why not the opposite? I think we need some reasoning here.}
% \zhi{I don't have a specific answer for this.}
% \nic{
% some candidate reasonings:
% \begin{enumerate}
% \item It's TCP and we are not that latency-aware in most use cases, while bandwidth is commonly demanded
% \item queuing delay is usually several magnitude lower than RTT anyways. also I believe there are prior literature claims TCP experience are only hurted after RTT is over 1 sec or so.
% \item pretty unrealisticly can queuing delay increase lead to a memory exhaustion
% \item realistically queuing delay is only indirectly affected by CC not a thing CC controls, one trying to attack for increasing queuing delay should probably
% look at queuing displine policies (e.g. fq, cake, ...)
% \item in shared hosting / cloud / multi-tanent env, bandwidth utilization directly affects profitability: while customers' may be metered for amount of data transferred, in most cases service providers are paying for a fixed bandwidth.
% \end{enumerate}
% to sum the impact to service quality may be rather limited comparing with targetting utilization.
% }
To reflect this idea in a new reward function, let $U_t$ denote the utilization at time step $t$. We define the queuing delay as the difference between the smoothed RTT and the minimum RTT:
\begin{equation}
d_t = RTT_t - RTT_{\min}.
\end{equation}
To improve the robustness against transient fluctuations while retaining responsiveness to instantaneous changes, we maintain a short history of recent queuing delays over a sliding window of size $H$. Based on this history, we compute a local average delay and a near-instantaneous delay estimate:
\begin{equation}
\bar{d}_t = \frac{1}{H} \sum_{i=t-H+1}^{t} d_i, \quad
\tilde{d}_t = \frac{1}{K} \sum_{i=t-K+1}^{t} d_i,
\end{equation}
where both $H$ and $K$ are small constants (e.g., $H=5$, $K=1$). Here, $\bar{d}_t$ captures short-term averaged behavior over the recent sliding window, while $\tilde{d}_t$ reflects the most recent observed delay. This design avoids sensitivity to noise while still enabling the reward to react quickly to current network conditions.
We then define a delay-based penalty term $R^{\text{delay}}_t$ using a threshold $\tau$:
\begin{equation}
R^{\text{delay}}_t =
\begin{cases}
-\alpha, & \text{if } \bar{d}_t < \tau \ \text{and} \ \tilde{d}_t < \tau, \\
0, & \text{otherwise},
\end{cases}
\end{equation}
where $\alpha > 0$ is a constant penalty coefficient. The threshold $\tau$ is derived from a baseline performance. Here, the baseline refers to the original performance of the algorithm without manipulation.
% \gang{This part needs to be moved to the other side.}
% Specifically, we generate bandwidth traces that satisfy the same average absolute slope constraint, but with randomly sampled variations, and evaluate the resulting queuing delay of the target policies.
The average queuing delay under baseline is $\tau$, which reflects the CC's performance in a non-adversarial setting. This ensures that the adversarial agent is required to reach at least this baseline level of delay before optimizing for utilization degradation.
Finally, the overall adversarial reward is defined as:
\begin{equation}
R^{\text{env}}_t = -U_t + R^{\text{delay}}_t
\end{equation}
This design encourages the adversarial agent to operate in a regime where queuing delay has already reached a predefined threshold, and then primarily optimizes for degradation in utilization. The lower-bound condition ($\bar{d}_t < \tau$ and $\tilde{d}_t < \tau$) prevents the agent from exploiting low-delay scenarios, where reduced utilization could be achieved without inducing meaningful congestion. By enforcing this constraint, we effectively decouple the impact of the utilization-delay trade-off, and the
% queuing delay is required to remain above a baseline level, and utilization becomes the primary metric for evaluation.
resulting changes in utilization can be interpreted unambiguously as performance degradation.
% \begin{tcolorbox}[width=\linewidth, colback=white!95!black, boxrule=0.5pt, left=2pt,right=2pt,top=1pt,bottom=1pt]
% \stepcounter{remark}
% {\bf Remark \arabic{remark}:}
% {
% Without explicit control over optimization direction, multi-objective optimization results become difficult to interpret due to inherent trade-offs; we address this by optimizing one objective while treating the other as a constraint.
% }
% \end{tcolorbox}
% rather than a byproduct of natural trade-offs between competing objectives.
% \insight{The inherent trade-off between utilization and delay obscures interpretation; by treating delay as a constraint and optimizing utilization, we decouple this trade-off and enable unambiguous evaluation of performance degradation.}
% These results highlight a fundamental challenge in interpretation due to the inherent trade-off between utilization and queuing delay. A congestion control policy can reduce queuing delay by behaving more conservatively, which naturally leads to lower utilization. As a result, it becomes unclear whether an observed change reflects true performance degradation or simply a natural trade-off inherent to congestion control. For example, while BBR and Vegas experience significant drops in utilization under adversarial perturbations, they simultaneously achieve lower queuing delay. This makes it difficult to determine whether their performance has worsened or merely shifted to a different operating point.
% \insight{Feature-level perturbation isolates signal dependence, providing a controlled setting to evaluate how sensitive a policy is to noise and adversarial perturbations on individual inputs.}
\subsection{Evaluation Setup}
% \gang{We may need to say a few words about why these protocols are considered? I feel the protocol selection should happen in the evaluation section later, not here. Here we should focus on a generic methodology.}
We apply the testing framework against a representative set of congestion controllers (CCs), including two learning-based policies (Orca and Canopy) and two non-learning-based designs (BBR and TCP Vegas). These protocols are selected because they all rely on minimum RTT estimation as part of their control logic. This shared dependency enables meaningful feature-level perturbation and allows us to directly compare their sensitivity to this input signal.
% For each CC, we first train the adversarial agent against them by \gang{XXXX add a bit more details about the training, e.g., say a few things about training data}.
For each CC, we first use reinforcement learning to train an adversarial agent in an online manner, where the agent interacts with the running protocol, dynamically manipulating the input feature (i.e., minimum RTT) at each RL step. After training, we then run the trained adversarial agent against the CC during runtime.
%%%%%%%%% I commented this out because it is simply repeating the previous section %%%%%%%%%%%%%
% We evaluate the adversarial agent after it has been trained using the reinforcement learning framework described above. During evaluation, the interaction protocol remains the same as in training. At each time step, the congestion control policy interacts with the network environment and produces control actions based on observed signals. The trained adversarial agent then observes the environment feedback and applies a perturbation to the target feature (i.e., the minimum RTT estimate) according to its learning-based policy. The modified feature is subsequently fed into the congestion controller, thereby influencing its behavior in a closed-loop manner.
% All evaluations are conducted using the simulated trace dataset provided by Canopy.
% \gang{We don't have to emphasize these are synthetic traces. We can just cite the dataset. Of course, it would be better if we add real traces or mix them together.}
% To ensure stability and reproducibility of results, each experiment is repeated multiple times on the same trace, and we report the average performance metrics across runs.
We evaluate the impact of feature-level perturbations under two adversarial capability levels, corresponding to $\pm5\%$ and $\pm50\%$ perturbation ranges on the minimum RTT signal. Each setting includes two main conditions: \textit{Clean} (without attack), and \textit{Adversarial} (with attack). Additionally, we run a \textit{Noise} condition (i.e., random perturbation within the same range). This noise setting (with random perturbation), unfortunately, could not control the utility-delay trade-off, and thus the results are only presented in Appendix~\ref{sec:feature_level_noise} for reference.
All evaluations are conducted with the network trace dataset used by Canopy~\cite{yang2024canopy}.
% \gang{provide more information about the traces.}
The dataset contains 18 simulated traces and 3 real-world traces. The simulated traces are bandwidth traces used to emulate time-varying network conditions in Mahimahi, enabling controlled yet diverse evaluation of congestion control behavior under dynamically changing link capacities. The real-world traces are derived from cellular network measurements (e.g., commercial LTE networks), capturing realistic bandwidth variability observed in operational environments.
In this section, we mainly report our results on the simulated traces, as these traces cover more diverse and challenging scenarios. We also performed additional experiments on real-world traces in Appendix~\ref{sec:feature_level_real} --- the conclusions remain consistent. Here, we randomly select 4 traces for training the agent, and perform testing on the remaining 14 traces. To ensure stability and reproducibility, each experiment is repeated three times per trace, and we report performance metrics averaged across runs and traces.
Unless otherwise specified, we fix the remaining environment parameters, following the same configurations used in previous work~\cite{yang2024canopy}. In particular, the base link delay is set to $10\,\mathrm{ms}$, and the queue size is configured as twice the bandwidth-delay product (2 BDP).
% These settings provide a controlled and consistent environment for comparing the impact of adversarial perturbations across different congestion control policies.
%%%%%%%%%%%%%%%%%%%% old results %%%%%%%%%%%%%%%%%
% \subsection{Performance and Findings}
% Under small perturbations ($\pm5\%$), the results in Table~\ref{tab:feature_minrtt_005} show that all protocols maintain largely stable performance across the three settings. Orca and Canopy exhibit nearly identical utilization and queuing delay compared to the clean baseline, and TCP Vegas also shows minimal variation. BBR is the only protocol with noticeable deviation, where utilization drops by approximately $9\%$ under noise and $7\%$ under adversarial perturbations. Interestingly, this reduction in utilization is accompanied by a slight decrease in queuing delay (around $1$\,ms), suggesting a more conservative operating behavior. These trends are further illustrated in Figure~\ref{fig:feature005}, where all policies cluster closely in the utilization--delay space. Overall, this indicates that small perturbations in the minRTT signal—representing mild measurement noise or estimation bias—have limited impact on most congestion control algorithms.
% As the perturbation magnitude increases to $\pm50\%$, the impact becomes more pronounced, as shown in Table~\ref{tab:feature_minrtt_05}. Learning-based policies (Orca and Canopy) maintain relatively stable utilization, with differences within approximately $1\%$ compared to the clean baseline, but at the cost of increased queuing delay. Specifically, Orca’s delay increases from around $4$\,ms to $12$\,ms, while Canopy experiences a smaller increase of about $4$\,ms. In contrast, non-learning-based policies exhibit significantly different behavior. BBR shows a substantial drop in utilization from $81.46\%$ to $49.33\%$, while its queuing delay decreases from $6.04$\,ms to $1.52$\,ms. TCP Vegas follows a similar pattern, with utilization reduced by nearly $50\%$, accompanied by a decrease in delay from $2.91$\,ms to $1.52$\,ms. These results indicate that both BBR and Vegas shift toward a highly conservative regime under strong perturbations, sacrificing throughput to reduce congestion. Figure~\ref{fig:feature05} further highlights this divergence, where the spread of points becomes significantly larger compared to the small perturbation case.
% These results highlight a fundamental challenge in interpretation due to the inherent trade-off between utilization and queuing delay. A congestion control policy can reduce queuing delay by behaving more conservatively, which naturally leads to lower utilization. As a result, it becomes unclear whether an observed change reflects true performance degradation or simply a natural trade-off inherent to congestion control. For example, while BBR and Vegas experience significant drops in utilization under adversarial perturbations, they simultaneously achieve lower queuing delay. This makes it difficult to determine whether their performance has worsened or merely shifted to a different operating point.
\begin{table}[t]
\centering
% \small
\begin{tabular}{llccc}
\toprule
\textbf{Model} & \textbf{Settings} & \textbf{Util. ($\downarrow$)} & \textbf{Delay ($\uparrow$)} & \textbf{P95 delay ($\uparrow$)} \\
\midrule
& Clean & 81.36 & 4.31 & 12.51 \\
Orca
& Adversarial & 80.48 & 4.50 & 19.00 \\
\midrule
& Clean & 94.32 & 19.21 & 45.53 \\
Canopy
& Adversarial & 94.22 & 20.30 & 46.61 \\
\midrule
& Clean & 81.46 & 6.04 & 22.17 \\
BBR
& Adversarial & 76.94 & 6.65 & 26.04 \\
\midrule
& Clean & 70.94 & 2.91 & 6.45 \\
Vegas
& Adversarial & 70.82 & 2.97 & 6.50 \\
\bottomrule
\end{tabular}
\caption{Feature-space manipulation results under minRTT perturbation ($[0.95, 1.05]$).
% In this experiment, the new reward is applied to maintain the queuing delay.
% \gang{Do we need to include Cubic since it is not affected by minRTT?}
}
\label{tab:feature_minrtt_005_newreward}
\end{table}
\begin{table}[t]
\centering
% \small
\begin{tabular}{llccc}
\toprule
\textbf{Model} & \textbf{Settings} & \textbf{Util. ($\downarrow$)} & \textbf{Delay ($\uparrow$)} & \textbf{P95 delay ($\uparrow$)} \\
\midrule
Orca & Clean & 81.36 & 4.31 & 12.51 \\
& Adversarial & 73.67 & 4.23 & 16.80 \\
\midrule
Canopy & Clean & 94.32 & 19.21 & 45.53 \\
& Adversarial & 91.70 & 22.22 & 49.47 \\
\midrule
BBR & Clean & 81.46 & 6.04 & 22.17 \\
& Adversarial & 66.78 & 6.25 & 26.78 \\
\midrule
Vegas & Clean & 70.94 & 2.91 & 6.45 \\
& Adversarial & 60.33 & 2.74 & 4.42 \\
\bottomrule
\end{tabular}
\caption{Feature-space manipulation results under minRTT perturbation ($[0.5, 1.5]$). }
% In this experiment, the new reward is applied to maintain the queuing delay.}
\label{tab:feature_minrtt_05_newreward}
\end{table}
\begin{figure}[t]
\centering
\begin{minipage}[t]{0.4\textwidth}
\includegraphics[width=0.99\textwidth]{feature_minrtt_005_all_policies_newreward.pdf}
\end{minipage}
\caption{
Feature-level adversarial testing results with 5\% adversarial capacity (new reward applied).
% \gang{Can we marge the two figures, since the figures look too empty for now. Also, can we remove 95 queuing delay from the figures, and just let readers read the tables?}
}
\label{fig:feature005_newreward}
\vspace{-0.12in}
\end{figure}
\begin{figure}[t]
\centering
\begin{minipage}[t]{0.4\textwidth}
\includegraphics[width=0.99\textwidth]{feature_minrtt_05_all_policies_newreward.pdf}
\end{minipage}
\caption{
Feature-level adversarial testing results with 50\% adversarial capacity (new reward applied).
}
\label{fig:feature05_newreward}
\vspace{-0.12in}
\end{figure}
\subsection{Performance and Findings}
The results are presented in Tables~\ref{tab:feature_minrtt_005_newreward} and~\ref{tab:feature_minrtt_05_newreward}.
We start by verifying whether the queuing delay constraint is consistently enforced. As shown in Tables~\ref{tab:feature_minrtt_005_newreward} and~\ref{tab:feature_minrtt_05_newreward}, the queuing delay under adversarial settings is no lower than (or comparable to) the clean baseline across both perturbation ranges. This confirms that the reward function helps to eliminate the ambiguity introduced by the utilization–delay trade-off, allowing us to isolate utilization degradation as the primary attack target.
Under small perturbations (±5\%), the overall trends remain consistent with our earlier observations.
% \gang{Zhi: since I commented out the old text, you might want to bring some of your old discussion back here (currently in the comment, you should be able to find them). Currently, I feel the result discussion is somewhat too brief.}
Figure~\ref{fig:feature005_newreward} shows that Orca, Canopy, and TCP Vegas maintain utilization levels very close to their clean baselines, indicating limited sensitivity to mild feature corruption. In contrast, BBR exhibits a modest degradation of around 5\%, suggesting a higher sensitivity even under relatively small perturbations.
As the adversarial capability increases to ±50\%, the differences across policies become more pronounced. From Table~\ref{tab:feature_minrtt_05_newreward}, Orca experiences an approximately 8\% drop in utilization, while Canopy shows a smaller degradation of around 3\%. In comparison, non-learning-based policies are more significantly affected: BBR suffers a substantial reduction of about 15\%, and TCP Vegas shows a degradation of roughly 10\%. These trends are further illustrated in Figure~\ref{fig:feature05_newreward}, where the separation between learning-based and non-learning-based policies becomes clearly visible when delay is controlled.
Overall, these results highlight that different CCs rely on shared input signals in fundamentally different ways, leading to heterogeneous sensitivity under adversarial perturbations. In particular, under feature-level manipulation of the minimum RTT signal, the learning-based policies (Orca and Canopy) demonstrate stronger robustness than the selected non-learning-based designs (BBR and TCP Vegas), primarily by maintaining stable utilization even under large perturbations. More broadly, the feature-level adversarial testing framework provides a controlled and systematic means to isolate and stress specific input dependencies, enabling developers to pinpoint vulnerabilities and better understand the robustness characteristics of their designs.
\begin{tcolorbox}[width=\linewidth, colback=white!95!black, boxrule=0.5pt, left=2pt,right=2pt,top=1pt,bottom=1pt]
\stepcounter{remark}
{\bf Remark \arabic{remark}:}
{
Under feature-level manipulation, most CCs remain robust to small perturbations (5\% of adversarial noise). As we increase the perturbation magnitude, learning-based CCs exhibit a higher level of robustness than non-learning-based CCs.
}
\end{tcolorbox}
% \gang{TODO for Zhi: improve the result discussion and maybe add a takeaway textbox.}
% Tables~\ref{tab:feature_minrtt_005_newreward} and~\ref{tab:feature_minrtt_05_newreward} summarize the impact of feature-level perturbations on the evaluated CCs under different adversarial capabilities. Each table includes three evaluation settings: a clean setting with no manipulation, a noise setting where random perturbations with the same magnitude as the adversarial budget are applied to the minRTT signal, and an adversarial setting where a learning-based agent adaptively selects perturbations. When the perturbation range is limited to $\pm5\%$ (Table~\ref{tab:feature_minrtt_005}), we observe no obvious performance degradation across most algorithms. As shown in Figure~\ref{fig:feature005}, all policies maintain similar utilization and delay compared to their baseline behavior. BBR and Canopy exhibit slight increase in queuing delay under adversarial perturbations. Orca and TCP Vegas have small utilization degradation. Overall, both learning-based and non-learning-based policies demonstrate robustness to small measurement noise or mild estimation bias in the minimum RTT signal.
% As the perturbation magnitude increases to $\pm50\%$ (Table~\ref{tab:feature_minrtt_05}), the impact becomes more differentiated across policies. In Figure~\ref{fig:feature05}, we observe that both TCP Vegas and BBR experience significant utilization degradation under adversarial manipulation, indicating strong sensitivity to large distortions in minimum RTT estimation. In contrast, the learning-based policies (Orca and Canopy) maintain relatively stable utilization, with only moderate increases in queuing delay. This suggests that while learning-based approaches may adjust their behavior under perturbed inputs, they are less prone to severe throughput degradation compared to traditional designs that rely more directly on delay signals.
% Overall, these results highlight that different CCs rely on shared input signals in fundamentally different ways, leading to heterogeneous sensitivity under adversarial perturbations. In particular, under feature-level manipulation of the minimum RTT signal, the learning-based policies (Orca and Canopy) demonstrate stronger robustness than the selected non-learning-based designs (BBR and TCP Vegas), primarily by maintaining stable utilization even under large perturbations. More broadly, the feature-level adversarial testing framework provides a controlled and systematic means to isolate and stress specific input dependencies, enabling developers to pinpoint vulnerabilities and better understand the robustness characteristics of their designs.
% \insight{Most policies are robust to small noise, suggesting that typical measurement errors are not the primary source of instability. Under larger perturbations, robustness diverges: learning-based policies degrade more gracefully, whereas delay-driven non-learning-based designs are more vulnerable due to their direct reliance on RTT signals.}
% \begin{table}[t]
% \centering
% \small
% \begin{tabular}{llccc}
% \toprule
% \textbf{Model} & \textbf{Settings} & \textbf{Util. ($\downarrow$)} & \textbf{Delay ($\uparrow$)} & \textbf{P95 delay ($\uparrow$)} \\
% \midrule
% Orca & Clean & 81.36 & 4.31 & 12.51 \\
% & Adversarial & 78.59 & 4.66 & 13.26 \\
% \midrule
% Canopy & Clean & 94.32 & 19.21 & 45.53 \\
% & Adversarial & 94.28 & 20.71 & 45.86 \\
% \midrule
% BBR & Clean & 81.46 & 6.04 & 22.17 \\
% & Adversarial & 74.64 & 6.11 & 18.97\\
% \midrule
% Vegas & Clean & 70.94 & 2.91 & 6.45 \\
% & Adversarial & 61.12 & 2.38 & 4.48 \\
% \bottomrule
% \end{tabular}
% \caption{Feature-space manipulation results under minRTT perturbation ($[0.95, 1.05]$).
% % \gang{Do we need to include Cubic since it is not affected by minRTT?}
% }
% \label{tab:feature_minrtt_005}
% \end{table}
% \begin{table}[t]
% \centering
% \small
% \begin{tabular}{llccc}
% \toprule
% \textbf{Model} & \textbf{Settings} & \textbf{Util. ($\downarrow$)} & \textbf{Delay ($\uparrow$)} & \textbf{P95 delay ($\uparrow$)} \\
% \midrule
% Orca & Clean & 81.36 & 4.31 & 12.51 \\
% & Adversarial & 80.10 & 12.78 & 39.22 \\
% \midrule
% Canopy & Clean & 94.32 & 19.21 & 45.53 \\
% & Adversarial & 93.93 & 23.15 & 45.67\\
% \midrule
% BBR & Clean & 81.46 & 6.04 & 22.17 \\
% & Adversarial & 49.33 & 1.52 & 2.48 \\
% \midrule
% Vegas & Clean & 70.94 & 2.91 & 6.45 \\
% & Adversarial & 18.25 & 1.52 & 2.97 \\
% \bottomrule
% \end{tabular}
% \caption{Feature-space manipulation results under minRTT perturbation ($[0.5, 1.5]$).}
% \label{tab:feature_minrtt_05}
% \end{table}
% \begin{figure}[t]
% \centering
% \begin{minipage}[t]{0.47\textwidth}
% \includegraphics[width=0.99\textwidth]{feature_minrtt_005_all_policies.pdf}
% \end{minipage}
% \caption{
% Feature-level adversarial testing results with 5\% adversarial capacity.
% % \gang{Can we marge the two figures, since the figures look too empty for now. Also, can we remove 95 queuing delay from the figures, and just let readers read the tables?}
% }
% \label{fig:feature005}
% \vspace{-0.12in}
% \end{figure}
% \begin{figure}[t]
% \centering
% \begin{minipage}[t]{0.47\textwidth}
% \includegraphics[width=0.99\textwidth]{feature_minrtt_05_all_policies.pdf}
% \end{minipage}
% \caption{
% Feature-level adversarial testing results with 50\% adversarial capacity.
% }
% \label{fig:feature05}
% \vspace{-0.12in}
% \end{figure}
\section{Environment-level Adversarial Testing}
\label{sec:env_level}
While feature-level sensitivity analysis provides useful diagnostic insights, it relies on direct manipulation of policy inputs and therefore may not reflect realistic deployment scenarios. In practice, congestion control policies are rarely exposed to adversarially crafted feature values. Instead, performance degradation typically arises from adverse or rapidly changing network conditions, such as fluctuating available bandwidth, competing traffic, or changes in end-to-end delay caused by routing or queuing dynamics.
To evaluate robustness under more realistic conditions, we conduct environment-level adversarial testing. In this stage, the adversarial agent does not manipulate policy inputs directly. Instead, it influences external network conditions that indirectly shape the observations available to the congestion control policy. This setting preserves the integrity of the policy’s internal logic and measurement pipeline, while allowing the adversary to induce challenging operating conditions through plausible network dynamics.
\subsection{Environment-level Control Surfaces}
In environment-level experiments, the adversarial agent operates on network parameters that are external to the congestion control algorithm but fundamentally define the environment in which the policy operates. We consider two such parameters: available bandwidth and link delay. These variables are not directly controlled by the policy, yet they shape the end-to-end measurements (e.g., throughput and RTT) that drive congestion control decisions.
Bandwidth variation captures a wide range of real-world phenomena, including cross traffic, traffic engineering decisions, and contention at shared bottlenecks. Link delay variation reflects effects such as route changes, queue buildup at intermediate nodes, and dynamic path selection. Importantly, we do not assume the presence of a deliberate or intelligent attacker manipulating these variables. Instead, we treat them as environmental factors that can naturally fluctuate across runs, potentially creating challenging or adverse conditions for the controller. This perspective allows us to evaluate how policies behave under difficult but plausible network dynamics that may arise in practice. To simplify the analysis and isolate effects, we focus on bandwidth control in the following experiments and do not vary link delay.
% \gang{Have we done any link delay experiments? }
% \insight{Environment-level manipulation evaluates robustness without altering policy inputs, exposing failures that arise from the interaction between control logic and network dynamics that may occur in realistic, challenging network conditions.}
% \subsection{Adversarial Manipulation Constraints}
\subsection{Adversarial Agent Design}
\label{sec:env-design}
For environment manipulation, the adversarial agent follows the same high-level design (with the improved reward function) as the feature-level agent (Section~\ref{sec:feature-design}). The additional factors we need to considier are (1) whether and how to control the level of dynamics of the environment traces, and (2) how to establish a baseline reference for queuing delay (a constraint in the reward function, see Section~\ref{sec:feature-design}).
\para{Unconstrained Adversarial Budget.}
We start with a {\em naive design} where the adversarial agent has no constraint on the bandwidth dynamics. In this setting, the adversarial agent is allowed to set an arbitrary bandwidth value at each time step within a predefined range (e.g., $[1, 96]$ Mbps, consistent with the range observed in the Canopy trace datasets). In this case, the adversarial agent learns to generate bandwidth traces that severely degrade the performance of congestion control policies. Figure~\ref{fig:env_orca_unconstrained} shows an example trace generated under this naive design, targeting Orca. While the traces successfully force a bad performance of the targeting CC (i.e., low bandwidth utilization and high queuing delay), the bandwidth traces also exhibit highly unrealistic dynamics, frequently oscillating between the minimum and maximum values with abrupt changes across consecutive time steps. For example, the bandwidth may jump from $1$ Mbps to $96$ Mbps within a single step and immediately drop back to $1$ Mbps in the next. Such traces will trivially force a bad performance, and limited insights can be drawn from the adversarial testing.
\begin{figure}[t]
\centering
\begin{minipage}[t]{0.47\textwidth}
\includegraphics[width=0.99\textwidth]{down-orca-cubic-nonpdo_start96-nonpdo_test_full-10-20-160_2_orca_arbitrary.pdf}
\end{minipage}
\caption{
Example of an adversarially generated bandwidth trace targeting Orca under the unconstrained setting.
}
\label{fig:env_orca_unconstrained}
\vspace{-0.12in}
\end{figure}
% Such extreme and high-frequency fluctuations are unlikely to occur in real-world networks, where bandwidth variations are typically constrained by physical link properties, traffic patterns, and network control mechanisms.
% While these unconstrained traces are effective in degrading performance, they offer limited practical value. The resulting failures are largely expected under such extreme conditions and do not provide meaningful insights into the underlying behavior of congestion control algorithms. In particular, these traces do not help diagnose specific weaknesses in control logic or guide improvements in protocol design, as system designers are unlikely to consider such unrealistic scenarios in practice.
% the agent will generate traces
% and Figure~\ref{fig:env_cubic_unconstrained} in the Appendix,
% which illustrate representative traces targeting Orca and Cubic, respectively, the resulting performance is extremely poor, characterized by very low utilization and significantly inflated queuing delay. These results confirm that our adversarial framework is functioning as intended: given sufficient flexibility, the agent is able to identify input sequences that achieve its optimization objective and expose worst-case behavior.
% However, a closer inspection of the generated traces reveals a critical limitation. The adversarial bandwidth sequences exhibit highly unrealistic dynamics, frequently oscillating between the minimum and maximum values with abrupt changes across consecutive time steps. For example, the bandwidth may jump from $1$ Mbps to $96$ Mbps within a single step and immediately drop back to $1$ Mbps in the next. Such extreme and high-frequency fluctuations are unlikely to occur in real-world networks, where bandwidth variations are typically constrained by physical link properties, traffic patterns, and network control mechanisms.
% While these unconstrained traces are effective in degrading performance, they offer limited practical value. The resulting failures are largely expected under such extreme conditions and do not provide meaningful insights into the underlying behavior of congestion control algorithms. In particular, these traces do not help diagnose specific weaknesses in control logic or guide improvements in protocol design, as system designers are unlikely to consider such unrealistic scenarios in practice.
% This observation highlights an important challenge in adversarial testing: without appropriate constraints, the adversary may exploit trivial and unrealistic strategies that maximize performance degradation but fail to produce actionable insights.
% \insight{Without constraints, adversarial agents exploit unrealistic extremes, producing trivial worst-case failures that lack practical relevance or diagnostic value.}
% Without such constraints, the adversarial agent may produce extreme, rapidly oscillating patterns that frequently jump between the minimum and maximum allowable values. While such patterns can degrade performance, they are highly unrealistic and provide limited practical insight, as abrupt fluctuations of this nature are unlikely to occur in real networks.
% \gang{We could add a quick and short experiment in the evaluation section to demonstrate the point of arbitrary mode: we do not need to cover all protocols, just 1-2 to demonstrate the points. The argument is that it will lead to ``trivial'' solutions to lead to worse performance.}
\begin{table*}[t]
\centering
% \small
\setlength{\tabcolsep}{6pt}
\renewcommand{\arraystretch}{1.25}
\begin{tabular}{lcccccc}
\toprule
\textbf{Tested Policy} & \textbf{Orca} & \textbf{Cubic} & \textbf{Vegas} & \textbf{Canopy} & \textbf{Illinois} & \textbf{LP} \\
\midrule
Baseline (random) & 95.58\% / 12.65ms & 94.50\% / 12.26ms & 65.16\% / 2.23ms & 89.69\% / 17.06ms & 98.69\% / 15.40ms & 85.58\% / 6.77ms \\
\midrule
\multicolumn{7}{l}{\textbf{Target Policy of Adv Agent}} \\
\midrule
Orca & \underline{85.49\% / 61.54ms} & 95.37\% / 51.25ms & 66.68\% / 26.25ms & 87.67\% / 68.88ms & 95.84\% / 55.17ms & 90.52\% / 40.97ms \\
Cubic & 85.53\% / 72.42ms & \underline{86.79\% / 43.91ms} & 50.08\% / 17.23ms & 88.46\% / 79.48ms & 90.14\% / 46.07ms & 80.16\% / 33.24ms \\
Vegas & 90.88\% / 18.49ms & \textbf{75.68\% / 15.41ms} & \underline{\textbf{48.42\% / 7.48ms}} & 84.78\% / 20.08ms & \textbf{80.25\% / 15.67ms} & 61.81\% / 13.14ms \\
Canopy & 90.11\% / 15.61ms & 76.70\% / 13.14ms & 49.96\% / 7.36ms & \underline{\textbf{82.72\% / 17.76ms}} & 83.27\% / 14.87ms & \textbf{55.95\% / 10.36ms} \\
Illinois & \textbf{83.45\% / 22.41ms} & 81.04\% / 19.80ms & 55.41\% / 10.75ms & 85.37\% / 25.19ms & \underline{83.60\% / 20.78ms} & 72.82\% / 15.48ms \\
LP & 93.36\% / 17.38ms & 87.06\% / 14.16ms & 54.27\% / 5.61ms & 85.31\% / 20.46ms & 92.41\% / 17.39ms & \underline{75.40\% / 10.26ms} \\
\bottomrule
\end{tabular}
\caption{Environment-level adversarial testing results under bandwidth-trace manipulation. Budget=48 Mbps, bw\_range=[1,96]. The \underline{underlined} cells refer to the setting where the adversarial agent was trained and tested on the same CC. \textbf{Bold} cells denote the strongest attacks that degrade the bandwidth utilization of a tested CC in each column.}
\label{tab:env_bw}
\end{table*}
\para{Constrained Adversarial Budget.}
Motivated by the above observations, we impose explicit constraints on the adversary's ability to manipulate bandwidth traces. More specifically, we introduce a budget-based constraint that enforces temporal smoothness in the generated traces. Specifically, let $b_t$ denote the bandwidth at time step $t$. We define the \emph{average absolute slope} over a sliding window of size $k$ (including the current step) as:
\begin{equation}
S_t = \frac{1}{k} \sum_{i=t-k+1}^{t} \left| b_i - b_{i-1} \right|.
\end{equation}
This metric captures the average magnitude of bandwidth changes between consecutive time steps within the window.
We then enforce a smoothness budget $\delta$ such that the average absolute slope $S_t \leq \delta$ at all time steps. During trace generation, each new bandwidth value $b_t$ is selected to ensure that the resulting sequence satisfies this constraint. This design prevents unrealistic high-frequency oscillations while still allowing the adversarial agent to generate sufficiently diverse and challenging network conditions. This allows protocol developers to generate traces that better reflect plausible real-world dynamics and provide more meaningful insights into the robustness of congestion control policies.
% \para{Random Baseline.}
% Unlike feature-level manipulation, we don't have a clear baseline to compare against, since the agent directly manipulates the bandwidth traces. Here, we need to establish a baseline for queuing delay to act as the constraint for the reward function.
% \gang{Need to mention how you create the baseline performance of the queuing delay}
% \gang{You need to discuss the random baseline here. Write more clearly here.}
% Specifically, we generate bandwidth traces that satisfy the same average absolute slope constraint, but with randomly sampled variations, and evaluate the resulting queuing delay of the target policies.
\para{Random Baseline.}
Unlike feature-level manipulation, we don't have a clear baseline to compare against, since the agent directly manipulates the bandwidth traces. Here, we need to establish a baseline for queuing delay to act as the constraint for the reward function.
Our idea is to construct a {\em random baseline} by generating bandwidth traces that follow the same constraints as the adversarial setting, {\em but without any optimization}. At each time step, the bandwidth is randomly sampled within the same range (e.g., $[1, 96]$ Mbps). To ensure fairness and realism, these traces are subject to the same smoothness constraint, i.e., the average absolute slope $S_t \leq \delta$ is enforced during generation. This ensures that the random traces exhibit comparable temporal dynamics.
We generate 10 such random traces and evaluate each congestion controller on these traces, repeating each run three times to account for variability. The reported random baseline is obtained by averaging across all runs and traces. This baseline reflects typical performance under constrained but non-adversarial network variations, and provides a reference point for measure the effect of adversarial optimization.
\subsection{Evaluation Setup}
We first train a dedicated adversarial agent to manipulate each of the target CCs. During training, the agent learns to generate bandwidth sequences that degrade the performance of a specific CC under the reward design described earlier.
For evaluation, we first deploy the trained adversarial agent online. More specifically, while the target CC is running and interacting with the environment, the corresponding adversarial agent observes the current environment feedback at each time step and generates the bandwidth value for the next step according to its learned policy. The generated bandwidth values are bounded within a predefined range of 1~Mbps to 96~Mbps, which reflects the typical operating conditions observed in the training datasets. This constraint ensures that adversarial traces remain within a realistic regime while still allowing sufficient flexibility to construct challenging conditions.
We repeat this process to obtain multiple adversarially generated bandwidth traces. Among the traces of each CC, we select the one that results in the worst performance for the target CC, subject to the delay constraint. This trace is then used as a fixed test environment and applied to all other CCs to evaluate {\em transferability}. We report the average performance of each policy over three runs on the selected trace to ensure stability of the results.
As a baseline, we generate random bandwidth traces that satisfy the same smoothness and range constraints as the adversarial traces. Each policy is evaluated on these random traces, and we report the average performance across runs. This baseline reflects typical performance under non-adversarial but dynamic network conditions and serves as a reference point for measuring adversarial impact.
All other environment parameters remain consistent with the settings described earlier~\ref{sec:feature_level}.
\begin{figure}[t]
\centering
\begin{minipage}[t]{0.4\textwidth}
\includegraphics[width=0.99\textwidth]{env_space_2.pdf}
\end{minipage}
\caption{
Environment-level adversarial testing results. We only show adversarial traces that lead to the strongest utilization degradation. The star represents performance on the random baseline, and the circle represents the performance on the strongest adversarial traces.
}
\label{fig:env_space}
\vspace{-0.12in}
\end{figure}
\subsection{Performance and Findings}
Table~\ref{tab:env_bw} and Figure~\ref{fig:env_space} summarize the environment-level adversarial testing results under bandwidth-trace manipulation.
First, in Table~\ref{tab:env_bw}, the diagonal entries (underlined) represent the setting where the adversarial agent is trained against a target CC and evaluated on the same CC. We observe that all adversaries successfully achieve their objective. In each case, the generated bandwidth traces induce queuing delay above the predefined threshold while simultaneously degrading utilization compared to the random baseline. This confirms that the proposed reward design is effective in guiding the adversarial agent to identify challenging network conditions that expose performance weaknesses.
Interestingly, we find that targeted adversarial traces are not always the most effective in degrading the utilization of a given CC. As shown in Table~\ref{tab:env_bw}, the off-diagonal entries represent transferred attacks where a trace generated against one target CC is evaluated on other CCs.
% \nic{\st{is ``directly optmized'' (and ``transferred'') introduced or explained before/anywhere?} Interestingly... sentence should go after The off-diagonal ... and need another sentence to start like ``eval. of transfer attacks shows'' }
The worst-case utilization for each tested policy is highlighted in bold. We find that in several cases, these transferred traces result in even lower utilization than the directly optimized ones, while still maintaining queuing delay above the threshold. We also observe that these transferred cases often have lower queuing delay, which likely helps to lower the bandwidth utilization even more. Overall, the result suggests that adversarial conditions discovered for one policy can generalize and may expose greater vulnerabilities in others.
% \gang{we we have a potential explanation for this?}
% \zhi{This phenomenon can be attributed to two possible reasons. First, due to the delay constraint in the reward design, transferred traces may operate near the constraint boundary. While they satisfy the average delay requirement during evaluation, they can intermittently violate the threshold under a different policy during training, triggering stronger penalties. Second, the traces identified by reinforcement learning may correspond to local optima, as the agent may not fully explore the space or escape to more adversarial cases.}
Finally, we quantify the maximum utilization degradation under the delay constraint for each policy. Learning-based policies (Orca and Canopy) exhibit relatively smaller degradation, around $12\%$ and $7\%$, respectively. In contrast, non-learning-based policies such as Cubic, TCP Vegas, TCP Illinois, and TCP LP experience significantly larger degradation, about $19\%$, $17\%$, $18\%$, and $30\%$, respectively. Overall, these results indicate that, under the considered adversarial setting, learning-based congestion controllers demonstrate stronger robustness compared to traditional non-learning-based ones.
% \insight{Under our environment-level adversarial setting, including the imposed constraints on bandwidth dynamics, learning-based CCs exhibit stronger robustness, with smaller utilization degradation than non-learning-based designs.}
\begin{tcolorbox}[width=\linewidth, colback=white!95!black, boxrule=0.5pt, left=2pt,right=2pt,top=1pt,bottom=1pt]
\stepcounter{remark}
{\bf Remark \arabic{remark}:}
{
Under environment-level bandwidth manipulation, learning-based CCs are also more robust than non-learning-based CCs, exhibiting smaller worst-case degradation. Also, we observe that {\em transferred} adversarial traces sometimes have a stronger impact than the {\em targeted} traces on degrading the performance of the given CCs.
}
\end{tcolorbox}
% \gang{TODO: add a section to explain why learning-based protocols are more robust than non-learning-based protocols.}
\subsection{Why Learning-based Policies Are More Robust}
\label{sec:env_reason}
The results in the previous subsection show that learning-based policies are consistently harder to degrade under our adversarial testing framework, exhibiting stronger robustness than non-learning-based designs. A natural question is \emph{why}.
\begin{table}[t]
\centering
\begin{tabular}{lcc}
\toprule
\textbf{Policy} & \textbf{cwnd Smoothiness} & \textbf{cwnd Smoothines (log)} \\
\midrule
Orca & 6544.59 & 7.13 \\
% \midrule
Canopy & 7149.33 & 7.34 \\
\midrule
% \midrule
Cubic & 180.13 & 2.15 \\
% \midrule
Vegas & 157.85 & 2.28 \\
% \midrule
Illinois & 951.42 & 4.46 \\
% \midrule
LP & 900.11 & 4.33 \\
\bottomrule
\end{tabular}
\caption{Average cwnd Smoothiness different policies running on adversarial generated traces (Canopy targeted)}
\label{tab:cwnd_smoothiness}
\end{table}
\para{Smoothness in Action Space?}
One hypothesis is that non-learning-based, rule-driven policies rely on fixed thresholds and pre-defined control rules, which may lead to relatively discrete congestion window (cwnd) adjustments. In contrast, learning-based policies operate over a more flexible and continuous action space, potentially enabling smoother control behavior. To verify this hypothesis, we record the cwnd evolution of all policies when running on the same adversarially generated trace. For this analysis, we focus on the adversarial trace targeting Canopy since all policies under this trace experience noticeable utilization degradation. To quantify cwnd dynamics, we first adopt the average absolute slope metric defined earlier. Additionally, since some policies may produce extremely large cwnd values, we introduce a log-scaled version to mitigate the impact of magnitude:
\begin{equation}
S_t^{\log} = \frac{1}{k} \sum_{i=t-k+1}^{t}
\left| \frac{\log(b_i) - \log(b_{i-1})}{t_i - t_{i-1}} \right|.
\end{equation}
The results are summarized in Table~\ref{tab:cwnd_smoothiness}. Surprisingly, learning-based policies exhibit \emph{significantly larger} cwnd variation. For example, Orca and Canopy achieve values of 6544.59 and 7149.33, respectively, while all non-learning-based policies remain below 1000. A similar trend is observed under the log-scaled metric. These results {\em do not support} the initial hypothesis: learning-based policies do not have smoother changes in the congestion window; instead, they exhibit much more aggressive and dynamic cwnd adjustments.
\para{Responsiveness of CWND Adjustments?}
To further understand the differences between learning-based and non-learning-based CCs, we further examine the cwnd adjustments of these CCs under the same adversarial traces. Figure~\ref{fig:cwnd_cubic} and Figure~\ref{fig:cwnd_orca} show an example. Each figure compares the ingress and egress throughput curves (upper panel) with the cwnd evolution (lower panel). For Cubic (Figure~\ref{fig:cwnd_cubic}), the cwnd can only loosely follow the bandwidth dynamics, showing limited responsiveness to rapid changes. In contrast, Orca (Figure~\ref{fig:cwnd_orca}) exhibits a much stronger alignment: when the ingress/egress curves need a peak to match the burst in available bandwidth, the cwnd will correspondingly increase, and vice versa.
These observations indicate that the advantage of learning-based policies does not come from smoother control, but from their ability to perform timely cwnd adjustments (even if some adjustments are drastic). Their more flexible action space enables rapid adaptation to dynamic conditions, allowing them to better track bandwidth variations under adversarial traces, whereas rule-based designs remain constrained by their predefined control mechanisms.
\begin{figure}[t]
\centering
\begin{minipage}[t]{0.45\textwidth}
\includegraphics[width=0.99\textwidth]{ldos_cwnd_cubic.drawio.pdf}
\end{minipage}
\caption{
A comparison example between Cubic's egress and ingress curves and cwnd curve on adversarial-generated bandwidth traces (Canopy targeted).
}
\label{fig:cwnd_cubic}
\vspace{-0.12in}
\end{figure}
\begin{figure}[t]
\centering
\begin{minipage}[t]{0.45\textwidth}
\includegraphics[width=0.99\textwidth]{ldos_cwnd_orca.drawio.pdf}
\end{minipage}
\caption{A comparison example between Orca's egress and ingress curves and cwnd curve on adversarial-generated bandwidth traces (Canopy targeted). }
\label{fig:cwnd_orca}
\vspace{-0.12in}
\end{figure}
\begin{tcolorbox}[width=\linewidth, colback=white!95!black, boxrule=0.5pt, left=2pt,right=2pt,top=1pt,bottom=1pt]
\stepcounter{remark}
{\bf Remark \arabic{remark}:}
{
Based on our cwnd analysis, learning-based CCs do not rely on smoother cwnd adjustments; instead, they exhibit more aggressive (higher-variation) cwnd dynamics. This enables more responsive adaptation to dynamic network conditions, leading to improved robustness.
}
\end{tcolorbox}
% \begin{table*}[t]
% \centering
% \small
% \setlength{\tabcolsep}{6pt}
% \renewcommand{\arraystretch}{1.25}
% \begin{tabular}{lcccccc}
% \toprule
% \textbf{Tested Policy} & \textbf{Orca} & \textbf{Cubic} & \textbf{Vegas} & \textbf{C3} \\
% \midrule
% Baseline (random) & 91.36\% / 20.73ms & 93.35\% / 19.27ms & 64.57\% / 5.87ms & 85.82\% / 19.67ms \\
% Baseline (C3's syn) & 81.47\% / 4.32ms & 99.08\% / 22.51ms & 71.10\% / 2.88ms & 94.17\% / 19.17ms \\
% \midrule
% \multicolumn{5}{l}{\textbf{Target Policy of Adv}} \\
% \midrule
% Orca & 50.80\% / 38.43ms & 64.62\% / 47.33ms & 42.89\% / 28.73ms & 57.72\% / 53.53ms \\
% Cubic & 42.99\% / 33.63ms & 62.59\% / 34.77ms & 49.88\% / 29.03ms & 64.45\% / 35.90ms \\
% Vegas & 73.83\% / 3.63ms & 89.21\% / 4.00ms & 36.22\% / 1.8ms & 78.50\% / 3.40ms \\
% C3 & 68.61\% / 31.33ms & 73.46\% / 25.67ms & 45.62\% / 14.85ms & 76.62\% / 29.77ms \\
% \bottomrule
% \end{tabular}
% \caption{Environment-level adversarial testing results under bandwidth-trace manipulation. Each cell reports utilization and average delay (utilization / delay). Rows under ``Target Policy of Adv'' indicate the adversarial agent trained against the specified target policy and evaluated across all tested policies. Budget=50, bw\_range=[1,48]}
% \label{tab:env_bw_budget}
% \end{table*}
% \begin{table*}[t]
% \centering
% \small
% \begin{tabular}{lcccccc}
% \toprule
% \textbf{Policy} & \textbf{Orca} & \textbf{Cubic} & \textbf{Vegas} & \textbf{Canopy} & \textbf{Illinois} & \textbf{LP} \\
% \midrule
% cwnd
% Smoothiness & 6544.59 & 180.13 & 157.85 & 7149.33 & 951.42 & 900.11 \\
% \midrule
% cwnd Smoothiness (log) & 7.13 & 2.15 & 2.28 & 7.34 & 4.46 & 4.33 \\
% \bottomrule
% \end{tabular}
% \caption{Average cwnd Smoothiness different policies running on adversarial generated traces (C3 targeted)}
% \label{tab:cwnd_smoothiness}
% \end{table*}
\section{A Case Study of TCP LP}
\label{sec:case_study}
The previous section shows that adversarially generated bandwidth traces can significantly degrade utilization across different congestion control policies. Beyond quantitative evaluation, an important question is what insights can be extracted from these traces. In this section, we perform a case study on TCP LP to understand how specific algorithmic design choices interact with adversarially induced network dynamics.
\subsection{Overview of TCP LP}
TCP LP is a delay-based congestion control algorithm built on top of TCP Reno, with an additional early congestion detection mechanism. Importantly, TCP LP is designed to operate as a low-priority background flow: its goal is to utilize only excess bandwidth without interfering with coexisting standard TCP connections, and to yield quickly when congestion is detected.
% so as to avoid inducing backoff in competing flows. We first briefly review TCP Reno.
To understand TCP LP, we first describe how TCP Reno works. TCP Reno operates in three main phases: slow start, congestion avoidance, and fast recovery. When the congestion window ($cwnd$) is below the slow start threshold ($ssthresh$), it increases exponentially, adding one MSS per ACK, resulting in exponential growth per RTT. Once $cwnd \geq ssthresh$, TCP Reno enters congestion avoidance, where $cwnd$ increases linearly according to:
\begin{equation}
cwnd \leftarrow cwnd + \frac{\text{MSS}^2}{cwnd}.
\end{equation}
When three duplicate ACKs are received, the sender enters the fast recovery state. At this point, the congestion window is reduced by setting $ssthresh = cwnd/2$ and adjusting $cwnd$ accordingly. The sender retransmits the lost packet and continues transmitting new data upon receiving duplicate ACKs to keep the pipeline full. Once the retransmitted packet is acknowledged, $cwnd$ is set to $ssthresh$, and the sender exits fast recovery and resumes congestion avoidance.
TCP LP inherits all TCP Reno states but introduces an additional layer of control based on delay signals to proactively detect congestion. In addition to standard loss-based mechanisms, TCP LP continuously monitors one-way delay (OWD), which reflects the time packets spend in the network, and its smoothed version (SOWD) to reduce noise and transient fluctuations. By maintaining estimates of the minimum and maximum observed delay, TCP LP establishes a baseline for propagation delay and tracks deviations caused by queue buildup. The key intuition is that increasing delay indicates growing queues in the network, which often precedes packet loss; thus, TCP LP attempts to anticipate congestion before losses occur. When the smoothed delay exceeds a predefined threshold, TCP LP interprets this as early congestion and reacts proactively by reducing the congestion window, either by halving it or, in more severe cases, resetting it to a very small value (e.g., $cwnd=1$). This design allows TCP LP to yield bandwidth earlier than traditional loss-based algorithms to minimize queuing delay, but also makes it more sensitive to delay fluctuations, which can lead to conservative bandwidth usage under dynamic or adversarial network conditions.
\subsection{Behavior under Adversarial Traces}
Using the adversarially generated bandwidth traces (Figure~\ref{fig:lp_example}), we observe a recurring burst-like pattern in available capacity, where bandwidth gradually increases to a peak and is then followed by a sustained decrease to a lower level, repeating over time. This structured variation has a distinct impact on TCP LP’s behavior.
During the initial several bursts, TCP LP operates in the congestion avoidance phase. No packet loss is observed, and the delay signal (OWD/SOWD) does not exceed the early congestion threshold. As a result, the congestion window increases linearly following TCP Reno’s congestion avoidance rule. However, due to the rise-and-decline pattern of the bandwidth, the sending rate does not fully adapt to the capacity reduction phase following each peak. This mismatch causes packets to accumulate in the queue, leading to a gradual buildup of queuing delay across successive cycles.
\begin{figure}[t]
\centering
\begin{minipage}[t]{0.45\textwidth}
\includegraphics[width=0.99\textwidth]{down-orca-lp-nonpdo_start96-nonpdo_test_full-10-20-160_3.pdf}
\end{minipage}
\caption{
Visualization of one testing case of TCP LP.
}
\label{fig:lp_example}
\vspace{-0.12in}
\end{figure}
\begin{figure}[t]
\centering
\begin{minipage}[t]{0.45\textwidth}
\includegraphics[width=0.99\textwidth]{down-orca-cubic-nonpdo_start96-nonpdo_test_full-10-20-160_replay_lp_orca.pdf}
\end{minipage}
\caption{
Replaying the TCP LP adversarial trace on Orca.
}
\label{fig:lp_on_orca}
\vspace{-0.12in}
\end{figure}
Although each individual triangle does not immediately trigger TCP LP’s early congestion detection, the accumulated effect across multiple cycles causes both OWD and its smoothed version (SOWD) to increase progressively. Eventually, at a later triangle, the smoothed delay exceeds the predefined threshold. At this point, TCP LP interprets the condition as early congestion and triggers a proactive response, sharply reducing the congestion window (e.g., halving it), even though packet loss has not yet occurred.
This behavior highlights a key characteristic of TCP LP under such adversarial traces: the algorithm does not react to instantaneous bandwidth decreases within a single cycle, but instead responds to the cumulative delay buildup over time. As a result, TCP LP may delay its reaction until queuing effects become pronounced, leading to abrupt window reductions that are not directly aligned with the instantaneous available bandwidth.
\subsection{Insights}
This behavior reveals two key insights into TCP LP’s design under such dynamics. First, TCP LP relies on delay as a signal of congestion, where increasing delay is interpreted as evidence of queue buildup in the network. However, this signal reflects the accumulation of packets over time rather than the instantaneous availability of bandwidth. In the adversarial traces, when bandwidth increases, the queue does not immediately drain, and the measured delay remains elevated due to packets buffered during earlier mismatches between sending rate and capacity. As a result, TCP LP continues to interpret that the network is congested even during periods when additional bandwidth has become available. Instead of probing this newly available capacity, the protocol reacts based on the elevated delay signal, which effectively captures past conditions rather than the current state of the network. This leads to situations where TCP LP refrains from increasing its sending rate, even though the underlying capacity would permit further utilization.
Second, once the delay signal exceeds the early congestion threshold, TCP LP applies abrupt reductions to the congestion window. While this behavior is consistent with its design goal of yielding quickly to competing traffic, it can be overly conservative under dynamic conditions such as the burst-like patterns observed here. In these environments, capacity fluctuates frequently, and short-lived increases in bandwidth require timely and gradual adaptation to be effectively utilized. However, TCP LP’s sharp backoff interrupts this process. After that, TCP LP must rebuild its congestion window through slow linear growth---before it can reach the available bandwidth, another cycle of delay buildup triggers early congestion again, causing it to back off once more.
% Under adversarially structured traces, this repeated cycle becomes more apparent, where TCP LP backs off based on accumulated delay even after capacity has recovered, preventing it from reaching available bandwidth.
As a comparison, we replay the same adversarial trace on Orca (a learning-based CC), as shown in Figure~\ref{fig:lp_on_orca}. Under the same bandwidth trace, Orca achieves substantially higher utilization (93.28\%) by adapting more flexibly to the burst-like bandwidth dynamics.
Note that this is not to criticize the design of TCP LP. Instead, the results can help protocol developers to validate their design and help to inform practitioners regarding which scenarios a given protocol may not work well.
% This contrast highlights that the performance degradation observed in TCP LP is not inherent to the trace itself, but stems from its design—specifically, its reliance on delay-based inference and abrupt backoff behavior. Under dynamic conditions, this design causes TCP LP to react to stale congestion signals and miss opportunities to utilize newly available bandwidth, leading to persistent bandwidth underutilization.
\begin{tcolorbox}[width=\linewidth, colback=white!95!black, boxrule=0.5pt, left=2pt,right=2pt,top=1pt,bottom=1pt]
\stepcounter{remark}
{\bf Remark \arabic{remark}:}
{
Adversarial traces can be used to reveal failure modes of a congestion controller and analyze how the design choices affect its performance under challenging network conditions.
}
\end{tcolorbox}
% While TCP LP can perform well under typical network conditions, this behavior reveals a key limitation under adversarially generated traces. During congestion avoidance, TCP LP underutilizes available bandwidth because its linear growth is too conservative relative to the rate of bandwidth increase. Meanwhile, its early congestion mechanism, although designed to prevent queue buildup, reacts to cumulative delay rather than instantaneous bandwidth changes. As a result, delay can silently accumulate across cycles, and once the threshold is exceeded, TCP LP triggers early congestion and sharply reduces the congestion window, even when the available bandwidth may not yet be fully utilized.
% This combination of conservative probing and delayed but aggressive backoff leads to persistent underutilization. More broadly, it highlights how TCP LP’s reliance on delay signals can cause its control actions to become misaligned with instantaneous network conditions under structured or adversarial dynamics, exposing subtle interactions between control logic and evolving bandwidth patterns that are not easily observable under standard evaluation settings. Importantly, such adversarial traces provide concrete and reproducible scenarios that reveal why a CCA may perform poorly in specific cases, offering valuable guidance for developers to diagnose weaknesses and refine their designs.
% \gang{This is to push the argument to the next level. Is any of the ``insights'' previously unknown to the network expert? Or are we stating the obvious? I think people want to see some insights that the LP designer may not even realize, for example.}
% \zhi{
% One contributing factor is that TCP LP relies primarily on delay-based signals for congestion inference, without incorporating additional context such as throughput or capacity trends. This design naturally favors conservative behavior, enabling early avoidance of potential congestion. However, under dynamic network conditions with oscillatory capacity, this reliance on delay alone can limit the protocol’s ability to fully exploit available bandwidth, as it may react before the flow has an opportunity to utilize short-lived capacity increases.
% A second contributing factor lies in LP’s reaction mechanism, which applies relatively abrupt reductions in sending rate once early congestion is detected. While this design reinforces LP’s goal of yielding quickly to competing traffic, it can be overly conservative under dynamic conditions. In oscillatory environments, where congestion signals are triggered repeatedly during successive capacity-growth phases, such abrupt responses prevent the flow from maintaining momentum and adapting smoothly to available bandwidth. As a result, LP repeatedly backs off before reaching efficient utilization, amplifying the underutilization effect across bursts. This suggests that more moderate or staged responses to early congestion signals may improve performance by allowing the protocol to better track short-term capacity variations while still preserving its low-priority behavior under sustained congestion.
% }
% \zhi{false signal }
% \nic{I'm also in question to this section. It feels like a bit cheating, especially when the nature of its design goal is not clearly explained in the first place. Original LP is designed with a goal of never causing backoff on other connections not on LP policy, and just normally use another TCP connection alongside LP should be able to cut its utilization...
% in other words its existence is simply to minimize OTHER connections' queuing delay and maximize other's utilization. It's goal is always to utilize ``excess'' bandwidth, not aim to use full. Also we should be really careful on the claim ``While TCP LP can perform well under typical network conditions, this behavior reveals a key limitation under adversarially generated traces.'' It seems to me either you're running multiple connections all using LP, or your trace is not in a realistic multi-connection competing emulation. See a screenshot I send in Slack channel: it's designed to have LP connections fairly compete, while give strict priority to cross-traffic on other best-effort CCs.
% {\bf I feel a change of tone is needed.} Instead of critizing it underutilize bandwidth, we can say for example it's rather outdated and modern hardware no longer fits its original assumption: possible arguments:
% 1. you don't need such a significant backoff to allow prioritized nowadays; 2. modern internet has many api requests that is so small which the original design did not foresee and unable to adapt (then possibly callback to the argument that LCCs are more adaptive and reactive, being versatile to dynamic end-user networks. another maybe good argument can be LCCs make a unified default policy for all connections better and alleviated some needs of sysadmin/dveloper picking different CC algs for their specific use cases for each connection)
% }
% \zhi{can we argue even if the "excess" bw is like the bw trace in the plot, lp cannot well-utilize all of them. }
% \insight{Adversarial traces expose protocol-specific worst-case scenarios and reveal how performance degradation arises from underlying design choices. This enables a systematic path from stress testing to actionable insights for improving congestion control algorithms.}
\section{Adversarial Training}
\label{sec:adv_train}
Our analysis so far shows that adversarially generated traces can identify failure modes of non-learning-based CCs by revealing subtle mismatches between control logic and network dynamics. For learning-based policies, such traces provide another opportunity: by incorporating the challenging traces into protocol training, we expect to further improve the learning-based CCs.
For this experiment, we focus on Orac since Orac does not have any robustness-enhancing designs. Orca, by default, is trained on samples collected from relatively stable bandwidth environments, which explains why its policy suffers under adverse dynamics. In comparison, Canopy already has a built-in robustness module (e.g., formally verified learning with robustness as a desired property). We will explore whether we can improve Orac's robustness with an adversarial training strategy, to match or exceed that of Canopy.
% In this study, we focus on Orca as a representative learning-based CC.
\subsection{Experiment Setup}
To evaluate adversarial training, we modify Orca's original training pipeline to explicitly control the exposure of the agent to adversarial traces. Instead of training from scratch, we initialize from the pre-trained model and continue training with an augmented trace pool.
\para{Trace Integration.}
In the original Orca training setup, multiple agents are typically trained in parallel, each interacting with a fixed bandwidth trace. To better control the training distribution, we abstract this process into a unified sampling framework. Specifically, rather than binding each agent to a fixed trace, we maintain a shared trace pool and allow the training process to dynamically select traces. During training, the agent interacts with one selected trace at a time for a fixed number of steps to collect experience, after which a new trace may be selected. We incorporate adversarial traces into this pool alongside the original benign traces, enabling the agent to experience both types of network dynamics within a single training process. Importantly, aside from the composition of the trace pool, all other components of the original training framework remain unchanged. In particular, the reward function, model architecture, and optimization procedure are identical to those used in the original Orca training setups.
\para{Mixing Ratio.}
Within this unified framework, the mixing ratio is defined as the probability of selecting an adversarial trace when sampling from the trace pool. At each trace switch, an adversarial trace is chosen with probability $p$, while a benign trace is selected with probability $1-p$. This abstraction allows us to directly control how frequently the agent is exposed to adversarial conditions, independent of the number of traces or parallel agents. The mixing ratio $p$ is treated as a hyperparameter, and a detailed study of its impact is provided in Appendix~\ref{sec:mixratio}.
\subsection{Performance and Findings}
% We evaluate the effectiveness of adversarial training under the setup described above. In our experiments, we set the mixing ratio to $p = 0.2$, meaning that adversarial traces are used in 20\% of the training episodes. The results are summarized in Table~\ref{tab:adv_retrain}.
We evaluate the effectiveness of adversarial training under the setup described above. Based on an analysis of the mixing ratio (Appendix~\ref{sec:mixratio}), we set $p = 0.2$, which provides a good balance between robustness under adversarial conditions and performance on benign workloads. The results are shown in Table~\ref{tab:adv_retrain}.
Overall, adversarial training does not significantly degrade performance on benign environments. On both the simulated and real-world evaluation datasets~\cite{yang2024canopy}, the retrained policy achieves comparable utilization, with only a moderate increase in queuing delay. Similarly, performance on the random baseline remains largely unchanged, indicating that the policy preserves its general behavior under standard conditions.
In contrast, the improvement on the adversarial trace is substantial. The retrained policy achieves higher utilization (6\% improvement, from 85.49\% to 91.66\%) while reducing queuing delay compared to the original model. Also, the re-trained Orca even performs better than the Canopy, a robustness-enhanced protocol~\cite{yang2024canopy}.
This suggests that exposure to adversarial dynamics during training enables the policy to better anticipate and adapt to structured bandwidth variations.
% In this section, Orca improves utilization by over 6\% on adversarial traces while maintaining comparable performance on benign workloads.
% These results demonstrate that adversarial traces can serve as effective training signals for learning-based CCs. By incorporating a controlled fraction of such traces, it is possible to improve robustness to challenging network conditions without sacrificing performance on typical workloads.
\begin{tcolorbox}[width=\linewidth, colback=white!95!black, boxrule=0.5pt, left=2pt,right=2pt,top=1pt,bottom=1pt]
\stepcounter{remark}
{\bf Remark \arabic{remark}:}
{
Adversarially generated traces expose hard cases that are underrepresented in standard training data. By incorporating these traces during training, learning-based CCs can achieve improved robustness under challenging network conditions without sacrificing performance on normal workloads.
}
\end{tcolorbox}
\begin{table}[t]
\centering
\begin{tabular}{lllcc}
\toprule
\textbf{Model} &\textbf{Model} & \textbf{Settings} & \textbf{Util. ($\uparrow$)} & \textbf{Delay ($\downarrow$)} \\
\midrule
\multirow{8}{*}{Orca} & \multirow{2}{*}{Random Baseline}
& No Retrain & 95.58 & 12.65 \\
& & Adv Train & 94.65 & 12.86 \\
\cmidrule{2-5}
& \multirow{2}{*}{Synthetic Traces~\cite{yang2024canopy}}
& No Retrain & 94.11 & 14.18 \\
& & Adv Train & 95.12 & 21.03 \\
\cmidrule{2-5}
& \multirow{2}{*}{Real-world Traces~\cite{yang2024canopy}}
& No Retrain & 83.21 & 29.41 \\
& & Adv Train & 83.89 & 30.09 \\
\cmidrule{2-5}
& \multirow{2}{*}{Adversarial Trace}
& No Retrain & 85.49 & 61.54 \\
& & Adv Train & 91.66 & 57.83 \\
\midrule
Canopy & Adversarial Trace & No Retrain & 86.67 & 68.99 \\
\bottomrule
\end{tabular}
\caption{Before and after adversarial training for Orca. The metrics are now displayed from the defenders' perspective (e.g., higher is better for utility).
}
\label{tab:adv_retrain}
\end{table}
% \begin{figure}[t]
% \centering
% \begin{minipage}[t]{0.47\textwidth}
% \includegraphics[width=0.99\textwidth]{adv_retrain_orca.pdf}
% \end{minipage}
% \caption{
% Before and after adversarial training for Orca.
% % \gang{don't use line type to distinguish different settings}
% \gang{We should remove this figure}
% }
% \label{fig:adv_retrain}
% \vspace{-0.12in}
% \end{figure}
% \begin{figure}[t]
% \centering
% \begin{minipage}[t]{0.47\textwidth}
% \includegraphics[width=0.99\textwidth]{util_vs_mixrate.pdf}
% \end{minipage}
% \caption{
% Utilization after adversarial training with different mix rates.
% }
% \label{fig:util_vs_mixrate}
% \vspace{-0.12in}
% \end{figure}
% \begin{figure}[t]
% \centering
% \begin{minipage}[t]{0.47\textwidth}
% \includegraphics[width=0.99\textwidth]{delay_vs_mixrate.pdf}
% \end{minipage}
% \caption{
% Queuing delay after adversarial training with different mix rates. \gang{These two figures look too big and too empty, make them smaller, two small figures in a column would be OK}
% }
% \label{fig:delay_vs_mixrate}
% \vspace{-0.12in}
% \end{figure}
% \begin{figure}[t]
% \centering
% \begin{subfigure}[t]{0.235\textwidth}
% \centering
% \includegraphics[width=0.99\textwidth]{util_vs_mixrate.pdf}
% \caption{Utilization}
% \end{subfigure}
% ~
% \begin{subfigure}[t]{0.235\textwidth}
% \centering
% \includegraphics[width=0.99\textwidth]{delay_vs_mixrate.pdf}
% \caption{Queuing Delay}
% \end{subfigure}
% \caption{Utilization and queuing delay after adversarial training with different mix rates. \gang{These two figures look too big and too empty, make them smaller, two small figures in a column would be OK}
% }
% \label{fig:delay_vs_mixrate}
% \end{figure}
\section{Discussion and Conclusion}
We introduce \sysname{}, an adversarial testing framework for evaluating and comparing learning- and non-learning-based congestion controllers (CCs). At the feature level, we examine the sensitivity of CCs to perturbations on shared input signals. At the environment level, we generate adversarial traces to evaluate how controllers behave under challenging network conditions. Through our experiments, we observe that learning-based congestion controllers exhibit greater robustness than traditional human-designed controllers. Our analysis further shows that this robustness does not stem from smoother control behavior, but from the ability to adapt more responsively to dynamic network conditions. Finally, we show adversarial training can help further improve the robustness of learning-based CCs.
Future work may explore broader use cases of CCLab. In this paper, we demonstrate adversarial testing that targets bandwidth utilization degradation. However, other attack targets (e.g., against fairness, queuing delay) can be implemented into CCLab by simply changing the {\em reward function} of the adversarial agent. In addition, future work may use CCLab to perform adversarial tests on other input signals (e.g., acknowledgment statistics) to cover other categories of CCs (e.g., loss-based CCs). Finally, future testing may consider manipulating other network environment conditions, such as link delay. Our evaluation is focused on bandwidth manipulation as it has a direct impact on the congestion controller. Intuitively, link delay should not have the same impact, and future work is needed to explicitly compare their impacts on CCs' behaviors.
% our framework also supports perturbation of link delay. In our experiments, we primarily focus on bandwidth, as it has a more direct and pronounced impact on controller behavior. Delay perturbations are supported but tend to have a less immediate effect compared to bandwidth dynamics.
% Finally, our evaluation is conducted based on
% we note that our evaluation is conducted using a network emulator (Mahimahi), which, while widely used, does not fully capture the complexity of real-world Internet environments. Although the emulator enables controlled and reproducible experiments with real protocol implementations, it abstracts away many factors such as multi-hop interactions, heterogeneous cross traffic, and deployment-specific dynamics.
% Overall, non-learning-based CCs span a wide range of design philosophies, from loss-driven heuristics to delay-aware and model-based approaches. Their explicit control logic makes them interpretable and easier to analyze, but also exposes them to potential weaknesses when underlying assumptions are violated or when input signals are systematically perturbed. Unlike learning-based approaches, these designs rely on simple estimation rules and analytical relationships rather than data-driven training. Their effectiveness therefore depends on the accuracy of the estimated network state, which may degrade under dynamic or adversarial conditions.
% Overall, learning-based congestion control algorithms differ significantly in their learning paradigms, ranging from online RL and utility-driven optimization to offline data-driven learning and verification-guided training. While these approaches demonstrate strong empirical performance and adaptability, they often rely on complex models and implicit decision logic, making their behavior difficult to interpret and reason about.
% \subsection{Key Takeaways and Evaluation Paradigm}
% This work introduces \sysname{}, an adversarial testing framework that enables a two-level diagnosis of congestion control policies. At the feature level, we examine the sensitivity of controllers to perturbations on shared input signals, providing insight into how policies depend on specific inputs. At the environment level, we generate adversarial yet constrained network dynamics to evaluate how controllers behave under systematically challenging conditions.
% Through this process, we observe that learning-based congestion controllers exhibit stronger robustness than traditional designs. Our analysis further shows that this robustness does not stem from smoother control behavior, but from the ability to adapt more responsively to dynamic network conditions.
% More broadly, our framework highlights adversarial testing as a complementary evaluation paradigm to traditional benchmarking. While prior evaluations primarily focus on average-case performance under fixed or benign conditions, adversarial testing enables the exploration of worst-case yet plausible scenarios. This provides a more complete understanding of controller behavior and exposes failure modes that are otherwise difficult to identify.
% \subsection{On the Utilization--Delay Trade-off}
% In our design, we treat the trade-off between bandwidth utilization and queuing delay as a confounding factor that can obscure interpretation. Specifically, a reduction in utilization may be accompanied by a decrease in queuing delay, making it difficult to distinguish true performance degradation from a natural shift along the trade-off curve. To address this, we constrain queuing delay and focus on utilization degradation as the primary adversarial objective.
% We emphasize that this design choice reflects one particular perspective rather than a universal objective. In practice, different system designers may prioritize different operating points. For example, one may intentionally seek configurations that reduce utilization while minimizing delay, effectively moving toward a lower-utilization, lower-delay regime. Such behaviors can be readily induced by modifying the adversarial reward function.
% Our framework is flexible in this regard: by adjusting the reward design, one can target different regions of the utilization and delay trade-off space and study controller behavior under alternative objectives. As such, our current formulation should be viewed as a controlled setting for isolating degradation, rather than a restriction on the types of behaviors that can be explored.
% \subsection{Control Surfaces and Framework Flexibility}
% In this paper, we focus on minimum RTT perturbation for feature-level analysis and bandwidth manipulation for environment-level testing. These choices are motivated by their broad relevance and interpretability. Minimum RTT is a widely used signal across both learning-based and traditional controllers, while bandwidth dynamics directly influence achievable performance.
% However, the framework is not limited to these control surfaces. At the feature level, it can be extended to perturb other signals such as pacing rate, estimated delay, or acknowledgment statistics, and can also support joint manipulation of multiple features. We focus on a single signal in this work to enable clear interpretation of sensitivity. From an implementation perspective, feature-level manipulation is naturally supported for learning-based controllers, where input signals are explicitly exposed to the policy. In contrast, applying similar perturbations to non-learning-based controllers is more challenging, as their input processing is embedded within kernel implementations and may require modification of the protocol stack.
% At the environment level, our framework also supports perturbation of link delay. In our experiments, we primarily focus on bandwidth, as it has a more direct and pronounced impact on controller behavior. Delay perturbations are supported but tend to have a less immediate effect compared to bandwidth dynamics.
% \subsection{Scope of Congestion Controllers}
% Our evaluation focuses on a representative set of congestion control algorithms, including both learning-based and non-learning-based designs. While this set is not exhaustive, it covers key categories such as delay-based, loss-based, and hybrid approaches, as well as recent learning-based policies.
% Importantly, the goal of this work is not to provide a comprehensive benchmark across all possible controllers, but to demonstrate a general framework for robustness evaluation. The methodology can be directly extended to additional protocols and emerging designs.
% \subsection{Generalizability Beyond Congestion Control}
% Although this work focuses on congestion control, the underlying methodology is broadly applicable to other domains where learning-based policies interact with dynamic environments. As machine learning systems increasingly replace or augment traditional rule-based designs in areas such as operating systems and networked systems, a key question arises: do these learned policies introduce new failure modes under challenging conditions?
% Our two-level analysis provides a general approach to addressing this question. Feature-level perturbation can be used to probe sensitivity to input signals, while environment-level manipulation enables the construction of structured, adversarial scenarios that reveal weaknesses in closed-loop behavior. These ideas can be adapted to other domains with appropriate control surfaces and constraints, offering a systematic way to evaluate robustness beyond average-case performance.
% \subsection{Limitations of Emulation-Based Evaluation}
% Finally, we note that our evaluation is conducted using a network emulator (Mahimahi), which, while widely used, does not fully capture the complexity of real-world Internet environments. Although the emulator enables controlled and reproducible experiments with real protocol implementations, it abstracts away many factors such as multi-hop interactions, heterogeneous cross traffic, and deployment-specific dynamics.
% As a result, our findings should be interpreted as evidence under controlled yet realistic conditions, rather than a complete characterization of behavior in the wild. Extending adversarial testing to large-scale or real-world deployments remains an important direction for future work.
\newpage
\section*{Ethics Considerations}
\label{sec:ethics}
This work evaluates congestion control algorithms in a fully controlled and emulated environment using Mahimahi, based on publicly available network traces. No experiments are conducted on live systems, and no real users or production traffic are affected.
% As a result, this study does not introduce risks to network infrastructure, service availability, or end users.
The adversarial testing framework is designed for evaluation and diagnostic purposes, rather than enabling real-world attacks. Its purpose is to identify failure modes and improve robustness.
% Feature-level perturbations model measurement noise or estimation errors, while environment-level manipulation reflects plausible variations in network conditions such as bandwidth fluctuation and delay.
% These settings align with naturally occurring dynamics and do not require access to or modification of deployed systems.
That being said, we acknowledge potential concerns regarding misuse of the framework. We believe the benefit of this work (enabling robustness testing for protocol developers) outweighs the potential risk. We will share our code with the broad community to facilitate result reproduction and protocol testing, and support future research. However, to prevent misuse, we will verify the identity of the code requester (e.g., based on their institutional email) before sharing the code.
% However, the proposed framework operates under bounded and abstracted perturbations and does not expose actionable attack strategies or system-specific vulnerabilities. Instead, its purpose is to identify failure modes and improve robustness.
Finally, this work does not involve the collection or analysis of personally identifiable information. The datasets used consist of network-level traces without user identities or sensitive content. Overall, our approach follows established ethical guidelines for ICT research, including minimizing harm and ensuring responsible use of experimental methods~\cite{kenneally2012menlo}.
%%%%%%%%
% \gang{TODO list:}
% \zhi{appendix: add soly perturbing link delay experiment}
% \zhi{5/4: related work section: add more stress-testing related papers}
% \zhi{write ethics section}
% \zhi{write discussion sections to compare with Giacomoni and Mazilu}
% \zhi{go back to related work and double check arguments on how we are different.}
\appendix
\section{Appendix}
\label{sec:appendix}
% \begin{figure}[t]
% \centering
% \begin{minipage}[t]{0.47\textwidth}
% \includegraphics[width=0.99\textwidth]{down-orca-cubic-nonpdo_start96-nonpdo_test_full-10-20-160_3_cubic_arbitrary.pdf}
% \end{minipage}
% \caption{
% Example of an adversarially generated bandwidth trace targeting Cubic under the unconstrained setting.
% }
% \label{fig:env_cubic_unconstrained}
% \vspace{-0.12in}
% \end{figure}
\subsection{Feature-Level Manipulation: Real-world Traces}
\label{sec:feature_level_real}
To validate the generality of our findings beyond Canopy simulated traces, we repeat the feature-level adversarial experiments on Canopy real-world traces. We also apply the redesigned reward that constrains queuing delay and isolates utilization degradation. The corresponding results are shown in Tables~\ref{tab:feature_minrtt_005_real_newreward} and~\ref{tab:feature_minrtt_05_real_newreward}. Under this setting, queuing delay in adversarial cases remains comparable to the clean baseline, enabling a more direct comparison of robustness.
Under small perturbations (±5\%), all policies exhibit minimal utilization changes, indicating robustness to mild noise. As perturbation strength increases to ±50\%, clearer differences emerge. Learning-based policies (Orca and Canopy) show relatively moderate degradation, while non-learning-based policies exhibit more pronounced sensitivity. In particular, TCP Vegas experiences the largest drop in utilization (11\%), and BBR also shows noticeable degradation (7\%), whereas learning-based designs maintain more stable performance.
Overall, the results on real-world traces are consistent with those observed on synthetic datasets. When controlling for the utilization–delay trade-off, learning-based CCAs continue to demonstrate stronger robustness to feature-level perturbations, while non-learning-based policies—especially those relying directly on RTT signals—are more susceptible to adversarial manipulation.
\subsection{Feature-Level Manipulation: Noise Setting}
\label{sec:feature_level_noise}
% In addition to learning-based adversarial perturbations, we consider a noise-based setting where the target feature (minimum RTT) is randomly perturbed within the same bounded range. Specifically, at each time step, the perturbation is uniformly sampled within ±5\% and ±50\% of the original value, matching the adversarial capability used in the main experiments. This setting serves as a baseline to understand the effect of unstructured perturbations without optimization.
% The results are summarized in Tables~\ref{tab:feature_minrtt_005_newreward_noise} and~\ref{tab:feature_minrtt_05_newreward_noise}, with corresponding visualizations in Figures~\ref{fig:feature005_newreward_noise} and~\ref{fig:feature05_newreward_noise}. Overall, random perturbations can still lead to noticeable performance change for all protocols. However, unlike adversarial perturbations, these effects are inconsistent and difficult to interpret.
% A key observation is that random perturbation inherently reflects the trade-off between utilization and queuing delay. In many cases, a decrease in utilization is accompanied by a reduction in queuing delay, indicating a shift toward a more conservative operating regime rather than a clear degradation in performance. Under ±50\% perturbation capacity, this behavior is particularly evident for BBR and TCP Vegas, both of which exhibit substantial utilization drops alongside reduced queuing delay. In contrast, learning-based policies such as Orca and Canopy largely maintain their utilization, but incur higher queuing delay. This reinforces the challenge discussed in the main text: changes in these metrics are often coupled, making it difficult to distinguish between true performance degradation and natural trade-off adjustments.
% From a geometric perspective, the noise-based setting can be viewed as randomly sampling operating points along the utilization–delay trade-off curve. Since the perturbation is unstructured, it does not provide control over the direction of movement in this space. As a result, the system may shift toward different regions: (i) lower utilization and lower delay (conservative behavior), (ii) higher utilization and higher delay (aggressive behavior), or (iii) in some cases, both worse utilization and higher delay. This variability leads to scattered outcomes, as illustrated in the corresponding figures.
In addition to adversarial perturbations, we consider a noise-based setting where the target feature (minimum RTT) is randomly perturbed within the same bounded range. Table~\ref{tab:noise_only} summarizes the results.
Unlike adversarial perturbations, random noise does not provide control over the utilization–delay trade-off. As a result, the system may shift toward different operating regimes, including (i) lower utilization with lower delay (more conservative behavior), (ii) higher utilization with higher delay (more aggressive behavior), or (iii) simultaneous degradation in both metrics. For example, under noise, BBR exhibits a noticeable drop in utilization accompanied by reduced queuing delay, reflecting a shift toward a more conservative operating point rather than a clear degradation. Overall, this lack of directional control leads to scattered and less interpretable outcomes.
% \begin{table}[t]
% \centering
% \small
% \begin{tabular}{llccc}
% \toprule
% \textbf{Model} & \textbf{Settings} & \textbf{Util. ($\downarrow$)} & \textbf{Delay ($\uparrow$)} & \textbf{P95 delay ($\uparrow$)} \\
% \midrule
% & Clean & 81.36 & 4.31 & 12.51 \\
% Orca & Noise & 83.75 & 4.64 & 13.82 \\
% & Adversarial & 78.59 & 4.66 & 13.26 \\
% \midrule
% & Clean & 94.32 & 19.21 & 45.53 \\
% Canopy & Noise & 93.35 & 19.10 & 45.54 \\
% & Adversarial & 94.28 & 20.71 & 45.86 \\
% \midrule
% & Clean & 81.46 & 6.04 & 22.17 \\
% BBR & Noise & 72.55 & 5.07 & 17.49\\
% & Adversarial & 74.64 & 6.11 & 18.97\\
% \midrule
% & Clean & 70.94 & 2.91 & 6.45 \\
% Vegas & Noise & 70.03 & 2.98 & 6.23 \\
% & Adversarial & 61.12 & 2.38 & 4.48 \\
% \bottomrule
% \end{tabular}
% \caption{Feature-space manipulation results under minRTT perturbation ($[0.95, 1.05]$).
% % \gang{Do we need to include Cubic since it is not affected by minRTT?}
% }
% \label{tab:feature_minrtt_005_noise}
% \end{table}
% \begin{table}[t]
% \centering
% \small
% \begin{tabular}{llccc}
% \toprule
% \textbf{Model} & \textbf{Settings} & \textbf{Util. ($\downarrow$)} & \textbf{Delay ($\uparrow$)} & \textbf{P95 delay ($\uparrow$)} \\
% \midrule
% & Clean & 81.36 & 4.31 & 12.51 \\
% Orca & Noise & 82.52 & 5.57 & 18.64 \\
% & Adversarial & 80.10 & 12.78 & 39.22 \\
% \midrule
% & Clean & 94.32 & 19.21 & 45.53 \\
% Canopy & Noise & 91.03 & 20.56 & 45.32 \\
% & Adversarial & 93.93 & 23.15 & 45.67\\
% \midrule
% & Clean & 81.46 & 6.04 & 22.17 \\
% BBR & Noise & 43.81 & 1.53 & 2.61 \\
% & Adversarial & 49.33 & 1.52 & 2.48 \\
% \midrule
% & Clean & 70.94 & 2.91 & 6.45 \\
% Vegas & Noise & 71.08 & 3.29 & 7.71\\
% & Adversarial & 18.25 & 1.52 & 2.97 \\
% \bottomrule
% \end{tabular}
% \caption{Feature-space manipulation results under minRTT perturbation ($[0.5, 1.5]$).}
% \label{tab:feature_minrtt_05_noise}
% \end{table}
% \begin{figure}[t]
% \centering
% \begin{minipage}[t]{0.47\textwidth}
% \includegraphics[width=0.99\textwidth]{No_use/feature_minrtt_005_all_policies_full.pdf}
% \end{minipage}
% \caption{
% Feature-level adversarial testing results with 5\% adversarial capacity.
% % \gang{Can we marge the two figures, since the figures look too empty for now. Also, can we remove 95 queuing delay from the figures, and just let readers read the tables?}
% }
% \label{fig:feature005_noise}
% \vspace{-0.12in}
% \end{figure}
% \begin{figure}[t]
% \centering
% \begin{minipage}[t]{0.47\textwidth}
% \includegraphics[width=0.99\textwidth]{No_use/feature_minrtt_05_all_policies_full.pdf}
% \end{minipage}
% \caption{
% Feature-level adversarial testing results with 50\% adversarial capacity.
% }
% \label{fig:feature05_noise}
% \vspace{-0.12in}
% \end{figure}
% \begin{table}[t]
% \centering
% \small
% \begin{tabular}{llccc}
% \toprule
% \textbf{Model} & \textbf{Settings} & \textbf{Util. ($\downarrow$)} & \textbf{Delay ($\uparrow$)} & \textbf{P95 delay ($\uparrow$)} \\
% \midrule
% & Clean & 81.36 & 4.31 & 12.51 \\
% Orca & Noise & 83.75 & 4.64 & 13.82 \\
% & Adversarial & 80.48 & 4.50 & 19.00 \\
% \midrule
% & Clean & 94.32 & 19.21 & 45.53 \\
% Canopy & Noise & 93.35 & 19.10 & 45.54 \\
% & Adversarial & 94.22 & 20.30 & 46.61 \\
% \midrule
% & Clean & 81.46 & 6.04 & 22.17 \\
% BBR & Noise & 72.55 & 5.07 & 17.49\\
% & Adversarial & 76.94 & 6.65 & 26.04 \\
% \midrule
% & Clean & 70.94 & 2.91 & 6.45 \\
% Vegas & Noise & 70.03 & 2.98 & 6.23 \\
% & Adversarial & 70.82 & 2.97 & 6.50 \\
% \bottomrule
% \end{tabular}
% \caption{Feature-space manipulation results under minRTT perturbation ($[0.95, 1.05]$). In this experiment, the new reward is applied to maintain the queuing delay.
% % \gang{Do we need to include Cubic since it is not affected by minRTT?}
% }
% \label{tab:feature_minrtt_005_newreward_noise}
% \end{table}
% \begin{table}[t]
% \centering
% \small
% \begin{tabular}{llccc}
% \toprule
% \textbf{Model} & \textbf{Settings} & \textbf{Util. ($\downarrow$)} & \textbf{Delay ($\uparrow$)} & \textbf{P95 delay ($\uparrow$)} \\
% \midrule
% & Clean & 81.36 & 4.31 & 12.51 \\
% Orca & Noise & 82.52 & 5.57 & 18.64 \\
% & Adversarial & 73.67 & 4.23 & 16.80 \\
% \midrule
% & Clean & 94.32 & 19.21 & 45.53 \\
% Canopy & Noise & 91.03 & 20.56 & 45.32 \\
% & Adversarial & 91.70 & 22.22 & 49.47 \\
% \midrule
% & Clean & 81.46 & 6.04 & 22.17 \\
% BBR & Noise & 43.81 & 1.53 & 2.61\\
% & Adversarial & 66.78 & 6.25 & 26.78 \\
% \midrule
% & Clean & 70.94 & 2.91 & 6.45 \\
% Vegas & Noise & 71.08 & 3.29 & 7.71\\
% & Adversarial & 60.33 & 2.74 & 4.42 \\
% \bottomrule
% \end{tabular}
% \caption{Feature-space manipulation results under minRTT perturbation ($[0.5, 1.5]$). In this experiment, the new reward is applied to maintain the queuing delay.}
% \label{tab:feature_minrtt_05_newreward_noise}
% \end{table}
% \begin{figure}[t]
% \centering
% \begin{minipage}[t]{0.47\textwidth}
% \includegraphics[width=0.99\textwidth]{No_use/feature_minrtt_005_all_policies_newreward_full.pdf}
% \end{minipage}
% \caption{
% Feature-level adversarial testing results with 5\% adversarial capacity (new reward applied).
% }
% \label{fig:feature005_newreward_noise}
% \vspace{-0.12in}
% \end{figure}
% \begin{figure}[t]
% \centering
% \begin{minipage}[t]{0.47\textwidth}
% \includegraphics[width=0.99\textwidth]{No_use/feature_minrtt_05_all_policies_newreward_full.pdf}
% \end{minipage}
% \caption{
% Feature-level adversarial testing results with 50\% adversarial capacity (new reward applied).
% }
% \label{fig:feature05_newreward_noise}
% \vspace{-0.12in}
% \end{figure}
% \begin{table}[t]
% \centering
% \small
% \begin{tabular}{llccc}
% \toprule
% \textbf{Model} & \textbf{Settings} & \textbf{Util. ($\downarrow$)} & \textbf{Delay ($\uparrow$)} & \textbf{P95 delay ($\uparrow$)} \\
% \midrule
% & Clean & 83.21 & 29.41 & 81.67 \\
% Orca & Noise & 84.51 & 29.39 & 81.83 \\
% & Adversarial & 83.24 & 28.03 & 78.69\\
% \midrule
% & Clean & 91.22 & 39.84 & 112.14 \\
% Canopy & Noise & 92.22 & 39.97 & 112.42 \\
% & Adversarial & 91.01 & 39.72 & 112.92 \\
% \midrule
% & Clean & 90.09 & 29.10 & 82.19\\
% BBR & Noise & 81.03 & 16.31 & 49.00\\
% & Adversarial & 71.03 & 14.57 & 46.80\\
% \midrule
% & Clean & 79.98 & 12.53 & 34.08\\
% Vegas & Noise & 75.27 & 11.68 & 31.67\\
% & Adversarial & 72.68 & 12.65 & 34.08 \\
% \bottomrule
% \end{tabular}
% \caption{Feature-space manipulation with real-world traces under minRTT perturbation ($[0.95, 1.05]$).
% }
% \label{tab:feature_minrtt_005_real}
% \end{table}
% \begin{table}[t]
% \centering
% \small
% \begin{tabular}{llccc}
% \toprule
% \textbf{Model} & \textbf{Settings} & \textbf{Util. ($\downarrow$)} & \textbf{Delay ($\uparrow$)} & \textbf{P95 delay ($\uparrow$)} \\
% \midrule
% & Clean & 83.21 & 29.41 & 81.67 \\
% Orca & Noise & 81.00 & 28.50 & 78.33 \\
% & Adversarial & 72.05 & 18.10 & 54.50 \\
% \midrule
% & Clean & 91.22 & 39.84 & 112.14 \\
% Canopy & Noise & 91.52 & 40.35 & 112.93 \\
% & Adversarial & 82.27 & 40.04 & 114.10\\
% \midrule
% & Clean & 90.09 & 29.10 & 82.19\\
% BBR & Noise & 72.09 & 15.98 & 50.17\\
% & Adversarial & 66.64 & 14.46 & 47.50\\
% \midrule
% & Clean & 79.98 & 12.53 & 34.08\\
% Vegas & Noise & 67.50 & 12.66 & 34.25\\
% & Adversarial & 44.87 & 8.80 & 25.17 \\
% \bottomrule
% \end{tabular}
% \caption{Feature-space manipulation with real-world traces under minRTT perturbation ($[0.5, 1.5]$).}
% \label{tab:feature_minrtt_05_real}
% \end{table}
% \begin{figure}[t]
% \centering
% \begin{minipage}[t]{0.47\textwidth}
% \includegraphics[width=0.99\textwidth]{feature_minrtt_005_all_policies_real.pdf}
% \end{minipage}
% \caption{
% Feature-level adversarial testing results with 5\% adversarial capacity.
% }
% \label{fig:feature005_real}
% \vspace{-0.12in}
% \end{figure}
% \begin{figure}[t]
% \centering
% \begin{minipage}[t]{0.47\textwidth}
% \includegraphics[width=0.99\textwidth]{feature_minrtt_05_all_policies_real.pdf}
% \end{minipage}
% \caption{
% Feature-level adversarial testing results with 50\% adversarial capacity.
% }
% \label{fig:feature05_real}
% \vspace{-0.12in}
% \end{figure}
% \begin{table}[t]
% \centering
% \small
% \begin{tabular}{llccc}
% \toprule
% \textbf{Model} & \textbf{Settings} & \textbf{Util. ($\downarrow$)} & \textbf{Delay ($\uparrow$)} & \textbf{P95 delay ($\uparrow$)} \\
% \midrule
% & Clean & 83.21 & 29.41 & 81.67 \\
% Orca & Noise & 84.51 & 29.39 & 81.83 \\
% & Adversarial & 83.17 & 31.63 & 87.21\\
% \midrule
% & Clean & 91.22 & 39.84 & 112.14 \\
% Canopy & Noise & 92.22 & 39.97 & 112.42 \\
% & Adversarial & 92.58 & 45.79 & 133.33\\
% \midrule
% & Clean & 90.09 & 29.10 & 82.19\\
% BBR & Noise & 81.03 & 16.31 & 49.37\\
% & Adversarial & 85.45 & 29.16 & 84.14\\
% \midrule
% & Clean & 79.98 & 12.53 & 34.08\\
% Vegas & Noise & 75.27 & 11.68 & 31.67\\
% & Adversarial & 77.88 & 12.44 & 39.17\\
% \bottomrule
% \end{tabular}
% \caption{Feature-space manipulation with real-world traces under minRTT perturbation ($[0.95, 1.05]$). In this experiment, the new reward is applied to maintain the queuing delay.
% }
% \label{tab:feature_minrtt_005_real_newreward}
% \end{table}
% \begin{table}[t]
% \centering
% \small
% \begin{tabular}{llccc}
% \toprule
% \textbf{Model} & \textbf{Settings} & \textbf{Util. ($\downarrow$)} & \textbf{Delay ($\uparrow$)} & \textbf{P95 delay ($\uparrow$)} \\
% \midrule
% & Clean & 83.21 & 29.41 & 81.67 \\
% Orca & Noise & 81.00 & 28.50 & 78.33 \\
% & Adversarial & 78.92 & 29.21 & 80.09 \\
% \midrule
% & Clean & 91.22 & 39.84 & 112.14 \\
% Canopy & Noise & 91.52 & 40.35 & 112.93 \\
% & Adversarial & 90.08 & 45.61 & 129.71\\
% \midrule
% & Clean & 90.09 & 29.10 & 82.19\\
% BBR & Noise & 72.09 & 15.98 & 50.17\\
% & Adversarial & 83.13 & 30.69 & 90.50 \\
% \midrule
% & Clean & 79.98 & 12.53 & 34.08\\
% Vegas & Noise & 67.50 & 12.66 & 34.25\\
% & Adversarial & 68.53 & 13.29 & 38.75 \\
% \bottomrule
% \end{tabular}
% \caption{Feature-space manipulation with real-world traces under minRTT perturbation ($[0.5, 1.5]$). In this experiment, the new reward is applied to maintain the queuing delay.}
% \label{tab:feature_minrtt_05_real_newreward}
% \end{table}
% \begin{figure}[t]
% \centering
% \begin{minipage}[t]{0.47\textwidth}
% \includegraphics[width=0.99\textwidth]{feature_minrtt_005_all_policies_real_newreward.pdf}
% \end{minipage}
% \caption{
% Feature-level adversarial testing results with 5\% adversarial capacity (new reward applied).
% }
% \label{fig:feature005_real_newreward}
% \vspace{-0.12in}
% \end{figure}
% \begin{figure}[t]
% \centering
% \begin{minipage}[t]{0.47\textwidth}
% \includegraphics[width=0.99\textwidth]{feature_minrtt_05_all_policies_real_newreward.pdf}
% \end{minipage}
% \caption{
% Feature-level adversarial testing results with 50\% adversarial capacity (new reward applied).
% }
% \label{fig:feature05_real_newreward}
% \vspace{-0.12in}
% \end{figure}
\begin{table}[t]
\centering
% \small
\begin{tabular}{llccc}
\toprule
\textbf{Model} & \textbf{Settings} & \textbf{Util. ($\downarrow$)} & \textbf{Delay ($\uparrow$)} & \textbf{P95 delay ($\uparrow$)} \\
\midrule
Orca & Clean & 83.21 & 29.41 & 81.67 \\
& Adversarial & 83.17 & 31.63 & 87.21\\
\midrule
Canopy & Clean & 91.22 & 39.84 & 112.14 \\
& Adversarial & 92.58 & 45.79 & 133.33\\
\midrule
BBR & Clean & 90.09 & 29.10 & 82.19\\
& Adversarial & 85.45 & 29.16 & 84.14\\
\midrule
Vegas & Clean & 79.98 & 12.53 & 34.08\\
& Adversarial & 77.88 & 12.44 & 39.17\\
\bottomrule
\end{tabular}
\caption{Feature-space manipulation with real-world traces under minRTT perturbation ($[0.95, 1.05]$). In this experiment, the new reward is applied to maintain the queuing delay.
}
\label{tab:feature_minrtt_005_real_newreward}
\end{table}
\begin{table}[t]
\centering
% \small
\begin{tabular}{llccc}
\toprule
\textbf{Model} & \textbf{Settings} & \textbf{Util. ($\downarrow$)} & \textbf{Delay ($\uparrow$)} & \textbf{P95 delay ($\uparrow$)} \\
\midrule
Orca & Clean & 83.21 & 29.41 & 81.67 \\
& Adversarial & 78.92 & 29.21 & 80.09 \\
\midrule
Canopy & Clean & 91.22 & 39.84 & 112.14 \\
& Adversarial & 90.08 & 45.61 & 129.71\\
\midrule
BBR & Clean & 90.09 & 29.10 & 82.19\\
& Adversarial & 83.13 & 30.69 & 90.50 \\
\midrule
Vegas & Clean & 79.98 & 12.53 & 34.08\\
& Adversarial & 68.53 & 13.29 & 38.75 \\
\bottomrule
\end{tabular}
\caption{Feature-space manipulation with real-world traces under minRTT perturbation ($[0.5, 1.5]$). In this experiment, the new reward is applied to maintain the queuing delay.}
\label{tab:feature_minrtt_05_real_newreward}
\end{table}
\begin{table}[t]
\centering
% \small
\setlength{\tabcolsep}{6pt}
\renewcommand{\arraystretch}{1.2}
\begin{tabular}{llccc}
\toprule
\textbf{Trace} & \textbf{Model} & \textbf{Util. ($\downarrow$)} & \textbf{Delay ($\uparrow$)} & \textbf{P95 delay ($\uparrow$)} \\
\midrule
\multirow{4}{*}{Simulated}
& Orca & 83.75 & 4.64 & 13.82 \\
& Canopy & 93.35 & 19.10 & 45.54 \\
& BBR & 72.55 & 5.07 & 17.49 \\
& Vegas & 70.03 & 2.98 & 6.23 \\
\midrule
\multirow{4}{*}{Real-world}
& Orca & 84.51 & 29.39 & 81.83 \\
& Canopy & 92.22 & 39.97 & 112.42 \\
& BBR & 81.03 & 16.31 & 49.37 \\
& Vegas & 75.27 & 11.68 & 31.67 \\
\bottomrule
\end{tabular}
\caption{Noise-based perturbation. All results correspond to the noise setting, where perturbations are randomly sampled within the same range as the adversarial budget.}
\label{tab:noise_only}
\end{table}
\begin{table}[t]
\centering
% \small
\begin{tabular}{ccccc}
\toprule
\textbf{Mixed Rate} & \multicolumn{2}{c}{\textbf{Random Baseline}} & \multicolumn{2}{c}{\textbf{Adv Train Performance}} \\
\cmidrule(lr){2-3} \cmidrule(lr){4-5}
& \textbf{Util. ($\downarrow$)} & \textbf{Delay ($\uparrow$)} & \textbf{Util. ($\downarrow$)} & \textbf{Delay ($\uparrow$)} \\
\midrule
0 & 95.58 & 12.65 & 85.49 & 61.54 \\
0.1 & 82.32 & 7.73 & 80.11 & 33.50 \\
0.2 & 90.82 & 8.70 & 91.66 & 57.83 \\
0.5 & 69.59 & 6.57 & 88.74 & 41.00 \\
0.8 & 33.72 & 2.97 & 90.35 & 39.03 \\
1 & 34.15 & 4.00 & 57.23 & 18.66 \\
\bottomrule
\end{tabular}
\caption{Adversarial training performance under different mixed rates.}
\label{tab:mixed_rate}
\end{table}
\subsection{Adv. Training Mixing Ratio Influence}
\label{sec:mixratio}
To further understand the role of adversarial exposure during training, we vary the mixing ratio $p$ and evaluate the resulting policies across both benign and adversarial environments. The utilization and queuing delay trends results are summarized in Table~\ref{tab:mixed_rate}.
We observe that the retrained policy exhibits significant performance variance across different mixing ratios. In particular, when $p$ increases from 0.2 to 0.8, utilization and queuing delay under adversarial traces improve consistently, indicating that more frequent exposure to adversarial dynamics enables the policy to better adapt to such conditions. However, this gain comes at the cost of degraded performance on benign environments. As the mixing ratio increases, the random baseline performance deteriorates, with utilization dropping substantially, suggesting that excessive adversarial exposure biases the policy toward overly conservative or specialized behaviors that do not generalize well to standard conditions.
An interesting phenomenon appears at low mixing ratios. When $p = 0.1$, both utilization and queuing delay decrease compared to the baseline. While the reduction in delay may appear beneficial, it is accompanied by a noticeable drop in utilization, indicating under-utilization of available bandwidth. This behavior can be attributed to the original reward function used in Orca training, which inherently balances the trade-off between utilization and queuing delay. Since the reward formulation remains unchanged during adversarial retraining, the policy may respond to limited adversarial exposure by adopting a more conservative strategy, reducing sending rates to avoid potential congestion signals. As a result, both metrics decrease simultaneously, reflecting a shift along the utilization-delay trade-off rather than a clear performance improvement.
Overall, these results highlight that the mixing ratio plays a critical role in shaping the learning-based policy. Moderate values (e.g., $p = 0.2$) provide a better balance, improving robustness to adversarial conditions while maintaining reasonable performance on benign traces. In contrast, overly small or large mixing ratios can lead to suboptimal behaviors, either failing to capture adversarial dynamics or overfitting to them at the expense of generalization.