Node-Time Conditional Prompt Learning In Dynamic Graphs
Extracted source text · arXiv 2405.13937
\documentclass{article} % For LaTeX2e
\usepackage{iclr2025_conference,times}
\usepackage{amsthm}
\usepackage{amsmath}
\usepackage{algorithm}
\usepackage{enumitem}
\usepackage{picinpar}
\usepackage{lineno}
\usepackage{graphicx}
% \usepackage{caption}
% \usepackage{subcaption}
% \usepackage{multirow}
\usepackage{algorithmicx}
\usepackage[noend]{algpseudocode}
\usepackage{subfigure}
\usepackage{multirow}
\usepackage{color}
\usepackage{balance}
\usepackage{enumitem}
\usepackage{hhline}
\usepackage[normalem]{ulem}
\usepackage{booktabs}
\usepackage{wrapfig}
\usepackage{cancel}
\usepackage{hyperref}
\usepackage{makecell}
\newtheorem{theorem}{Theorem}
\newtheorem{lemma}{Lemma}
\newtheorem{corollary}{Corollary}
\newtheorem{assumption}{Assumption}
\newtheorem{definition}{Definition}
\newcommand{\bL}{\ensuremath{\mathcal{L}}}
\newcommand{\bU}{\ensuremath{\mathcal{U}}}
\newcommand{\bS}{\ensuremath{\mathcal{S}}}
\newcommand{\bH}{\ensuremath{\mathcal{H}}}
\newcommand{\bW}{\ensuremath{\mathcal{W}}}
\newcommand{\bV}{\ensuremath{\mathcal{V}}}
\newcommand{\bG}{\ensuremath{\mathcal{G}}}
\newcommand{\bN}{\ensuremath{\mathcal{N}}}
\newcommand{\bE}{\ensuremath{\mathcal{E}}}
\newcommand{\bT}{\ensuremath{\mathcal{T}}}
\newcommand{\bY}{\ensuremath{\mathcal{Y}}}
\newcommand{\bM}{\ensuremath{\mathcal{M}}}
\newcommand{\bC}{\ensuremath{\mathcal{C}}}
\newcommand{\bP}{\ensuremath{\mathcal{P}}}
\newcommand{\bD}{\ensuremath{\mathcal{D}}}
\newcommand{\cls}{\text{cls}}
\newcommand{\str}{\text{str}}
\newcommand{\con}{\text{con}}
\newcommand{\fang}[1]{\textcolor{red}{#1}}
\newcommand{\yu}[1]{\textcolor{blue}{#1}}
\renewcommand{\vec}[1]{\ensuremath{\mathbf{#1}}}
\newcommand{\stitle}[1]{\vspace{1mm} \noindent {\bf #1}}
\newcommand{\eg}{{\it e.g.}}
\newcommand{\etal}{{\it et al.}}
\newcommand{\ie}{{\it i.e.}}
\newcommand{\etc}{{\it etc.}}
\newcommand{\wrt}{w.r.t. }
\newcommand{\vs}{{\it vs.}}
\newcommand{\tabincell}[2]{\begin{tabular}{@{}#1@{}}#2\end{tabular}}
%\newcommand{\method}[1]{\scalebox{0.9}{\textsf{#1}}}
\newcommand{\method}[1]{\textsc{#1}}
% \newcommand{\method}{}
\newcommand{\model}{\method{DyGPrompt}{}}
\newcommand{\concept}{\method{homophily sensitivity}{}}
\newcommand{\uconcept}{\method{homophily independence}{}}
\newcommand{\modelS}[1]{\method{SOTA-{#1}}}
\newcommand{\eat}[1]{}
\renewcommand{\algorithmicrequire}{\textbf{Input:}}
\renewcommand{\algorithmicensure}{\textbf{Output:}}
\newcommand\xingtong[1]{\textcolor{green}{#1}}
\newcommand\xingtongC[1]{\begin{CJK*}{UTF8}{gbsn}\textcolor{green}{[\textbf{Xingtong:} #1]}\end{CJK*}}
\newcommand{\stkout}[1]{\ifmmode\text{\sout{\ensuremath{#1}}}\else\sout{#1}\fi}
% Optional math commands from https://github.com/goodfeli/dlbook_notation.
\input{math_commands.tex}
\usepackage{hyperref}
\usepackage{url}
\title{Node-Time Conditional Prompt Learning in Dynamic Graphs}
% Authors must not appear in the submitted version. They should be hidden
% as long as the \iclrfinalcopy macro remains commented out below.
% Non-anonymous submissions will be rejected without review.
% \author{Antiquus S.~Hippocampus, Natalia Cerebro \& Amelie P. Amygdale \thanks{ Use footnote for providing further information
% about author (webpage, alternative address)---\emph{not} for acknowledging
% funding agencies. Funding acknowledgements go at the end of the paper.} \\
% Department of Computer Science\\
% Cranberry-Lemon University\\
% Pittsburgh, PA 15213, USA \\
% \texttt{\{hippo,brain,jen\}@cs.cranberry-lemon.edu} \\
% \And
% Ji Q. Ren \& Yevgeny LeNet \\
% Department of Computational Neuroscience \\
% University of the Witwatersrand \\
% Joburg, South Africa \\
% \texttt{\{robot,net\}@wits.ac.za} \\
% \AND
% Coauthor \\
% Affiliation \\
% Address \\
% \texttt{email}
% }
\author{Xingtong Yu\textsuperscript{\rm 1}\thanks{Co-first authors.}~~,
Zhenghao Liu\textsuperscript{\rm 2}$^*$,
Xinming Zhang\textsuperscript{\rm 2}\thanks{Corresponding authors.}~~,
Yuan Fang\textsuperscript{\rm 1}$^\dagger$\\
Singapore Management University\textsuperscript{\rm 1},
University of Science and Technology of China\textsuperscript{\rm 2}\\
\{xingtongyu, yfang\}@smu.edu.sg, salzh@mail.ustc.edu.cn, xinming@ustc.edu.cn
}
% The \author macro works with any number of authors. There are two commands
% used to separate the names and addresses of multiple authors: \And and \AND.
%
% Using \And between authors leaves it to \LaTeX{} to determine where to break
% the lines. Using \AND forces a linebreak at that point. So, if \LaTeX{}
% puts 3 of 4 authors names on the first line, and the last on the second
% line, try using \AND instead of \And before the third author name.
\newcommand{\fix}{\marginpar{FIX}}
\newcommand{\new}{\marginpar{NEW}}
\iclrfinalcopy % Uncomment for camera-ready version, but NOT for submission.
\begin{document}
\maketitle
\begin{abstract}
Dynamic graphs capture evolving interactions between entities, such as in social networks, online learning platforms, and crowdsourcing projects. For dynamic graph modeling, dynamic graph neural networks (DGNNs) have emerged as a mainstream technique. However, they are generally pre-trained on the link prediction task, leaving a significant gap from the objectives of downstream tasks such as node classification. To bridge the gap, prompt-based learning has gained traction on graphs, but most existing efforts focus on static graphs and neglect the evolution of dynamic graphs.
In this paper, we propose \model, a novel pre-training and prompt learning framework for dynamic graph modeling. First, we design \emph{dual prompts} to address the discrepancy in both task objectives and temporal variations across pre-training and downstream tasks. Second, we recognize that node and time patterns often characterize each other, and propose \emph{dual condition-nets} to model the evolving node-time patterns in downstream tasks.
Finally, we thoroughly evaluate and analyze \model\ through extensive experiments on four public datasets.
\end{abstract}
\input{sec-introduction}
\input{sec-related-work}
\input{sec-preliminary}
% \input{sec-theory}
\input{sec-model}
\input{sec-experiments}
\input{sec-conclusions}
% \clearpage
% \newpage
\section*{Acknowledgments}
This research / project is supported by the Ministry of Education, Singapore, under its Academic Research Fund Tier 2 (Proposal ID: T2EP20122-0041). Any opinions, findings and conclusions or recommendations expressed in this material are those of the author(s) and do not reflect the views of the Ministry of Education, Singapore.
\clearpage
\newpage
\bibliography{references}
\bibliographystyle{iclr2025_conference}
\newpage
% \appendix
% \section{Appendices}
% \renewcommand\thesubsection{\Alph{subsection}}
% \renewcommand\thesubsubsection{\thesubsection.\arabic{subsection}}
\appendix
\input{sec-appendix}
\end{document}
%%%%% NEW MATH DEFINITIONS %%%%%
\usepackage{amsmath,amsfonts,bm}
% Mark sections of captions for referring to divisions of figures
\newcommand{\figleft}{{\em (Left)}}
\newcommand{\figcenter}{{\em (Center)}}
\newcommand{\figright}{{\em (Right)}}
\newcommand{\figtop}{{\em (Top)}}
\newcommand{\figbottom}{{\em (Bottom)}}
\newcommand{\captiona}{{\em (a)}}
\newcommand{\captionb}{{\em (b)}}
\newcommand{\captionc}{{\em (c)}}
\newcommand{\captiond}{{\em (d)}}
% Highlight a newly defined term
\newcommand{\newterm}[1]{{\bf #1}}
% Figure reference, lower-case.
\def\figref#1{figure~\ref{#1}}
% Figure reference, capital. For start of sentence
\def\Figref#1{Figure~\ref{#1}}
\def\twofigref#1#2{figures \ref{#1} and \ref{#2}}
\def\quadfigref#1#2#3#4{figures \ref{#1}, \ref{#2}, \ref{#3} and \ref{#4}}
% Section reference, lower-case.
\def\secref#1{section~\ref{#1}}
% Section reference, capital.
\def\Secref#1{Section~\ref{#1}}
% Reference to two sections.
\def\twosecrefs#1#2{sections \ref{#1} and \ref{#2}}
% Reference to three sections.
\def\secrefs#1#2#3{sections \ref{#1}, \ref{#2} and \ref{#3}}
% Reference to an equation, lower-case.
\def\eqref#1{equation~\ref{#1}}
% Reference to an equation, upper case
\def\Eqref#1{Equation~\ref{#1}}
% A raw reference to an equation---avoid using if possible
\def\plaineqref#1{\ref{#1}}
% Reference to a chapter, lower-case.
\def\chapref#1{chapter~\ref{#1}}
% Reference to an equation, upper case.
\def\Chapref#1{Chapter~\ref{#1}}
% Reference to a range of chapters
\def\rangechapref#1#2{chapters\ref{#1}--\ref{#2}}
% Reference to an algorithm, lower-case.
\def\algref#1{algorithm~\ref{#1}}
% Reference to an algorithm, upper case.
\def\Algref#1{Algorithm~\ref{#1}}
\def\twoalgref#1#2{algorithms \ref{#1} and \ref{#2}}
\def\Twoalgref#1#2{Algorithms \ref{#1} and \ref{#2}}
% Reference to a part, lower case
\def\partref#1{part~\ref{#1}}
% Reference to a part, upper case
\def\Partref#1{Part~\ref{#1}}
\def\twopartref#1#2{parts \ref{#1} and \ref{#2}}
\def\ceil#1{\lceil #1 \rceil}
\def\floor#1{\lfloor #1 \rfloor}
\def\1{\bm{1}}
\newcommand{\train}{\mathcal{D}}
\newcommand{\valid}{\mathcal{D_{\mathrm{valid}}}}
\newcommand{\test}{\mathcal{D_{\mathrm{test}}}}
\def\eps{{\epsilon}}
% Random variables
\def\reta{{\textnormal{$\eta$}}}
\def\ra{{\textnormal{a}}}
\def\rb{{\textnormal{b}}}
\def\rc{{\textnormal{c}}}
\def\rd{{\textnormal{d}}}
\def\re{{\textnormal{e}}}
\def\rf{{\textnormal{f}}}
\def\rg{{\textnormal{g}}}
\def\rh{{\textnormal{h}}}
\def\ri{{\textnormal{i}}}
\def\rj{{\textnormal{j}}}
\def\rk{{\textnormal{k}}}
\def\rl{{\textnormal{l}}}
% rm is already a command, just don't name any random variables m
\def\rn{{\textnormal{n}}}
\def\ro{{\textnormal{o}}}
\def\rp{{\textnormal{p}}}
\def\rq{{\textnormal{q}}}
\def\rr{{\textnormal{r}}}
\def\rs{{\textnormal{s}}}
\def\rt{{\textnormal{t}}}
\def\ru{{\textnormal{u}}}
\def\rv{{\textnormal{v}}}
\def\rw{{\textnormal{w}}}
\def\rx{{\textnormal{x}}}
\def\ry{{\textnormal{y}}}
\def\rz{{\textnormal{z}}}
% Random vectors
\def\rvepsilon{{\mathbf{\epsilon}}}
\def\rvtheta{{\mathbf{\theta}}}
\def\rva{{\mathbf{a}}}
\def\rvb{{\mathbf{b}}}
\def\rvc{{\mathbf{c}}}
\def\rvd{{\mathbf{d}}}
\def\rve{{\mathbf{e}}}
\def\rvf{{\mathbf{f}}}
\def\rvg{{\mathbf{g}}}
\def\rvh{{\mathbf{h}}}
\def\rvu{{\mathbf{i}}}
\def\rvj{{\mathbf{j}}}
\def\rvk{{\mathbf{k}}}
\def\rvl{{\mathbf{l}}}
\def\rvm{{\mathbf{m}}}
\def\rvn{{\mathbf{n}}}
\def\rvo{{\mathbf{o}}}
\def\rvp{{\mathbf{p}}}
\def\rvq{{\mathbf{q}}}
\def\rvr{{\mathbf{r}}}
\def\rvs{{\mathbf{s}}}
\def\rvt{{\mathbf{t}}}
\def\rvu{{\mathbf{u}}}
\def\rvv{{\mathbf{v}}}
\def\rvw{{\mathbf{w}}}
\def\rvx{{\mathbf{x}}}
\def\rvy{{\mathbf{y}}}
\def\rvz{{\mathbf{z}}}
% Elements of random vectors
\def\erva{{\textnormal{a}}}
\def\ervb{{\textnormal{b}}}
\def\ervc{{\textnormal{c}}}
\def\ervd{{\textnormal{d}}}
\def\erve{{\textnormal{e}}}
\def\ervf{{\textnormal{f}}}
\def\ervg{{\textnormal{g}}}
\def\ervh{{\textnormal{h}}}
\def\ervi{{\textnormal{i}}}
\def\ervj{{\textnormal{j}}}
\def\ervk{{\textnormal{k}}}
\def\ervl{{\textnormal{l}}}
\def\ervm{{\textnormal{m}}}
\def\ervn{{\textnormal{n}}}
\def\ervo{{\textnormal{o}}}
\def\ervp{{\textnormal{p}}}
\def\ervq{{\textnormal{q}}}
\def\ervr{{\textnormal{r}}}
\def\ervs{{\textnormal{s}}}
\def\ervt{{\textnormal{t}}}
\def\ervu{{\textnormal{u}}}
\def\ervv{{\textnormal{v}}}
\def\ervw{{\textnormal{w}}}
\def\ervx{{\textnormal{x}}}
\def\ervy{{\textnormal{y}}}
\def\ervz{{\textnormal{z}}}
% Random matrices
\def\rmA{{\mathbf{A}}}
\def\rmB{{\mathbf{B}}}
\def\rmC{{\mathbf{C}}}
\def\rmD{{\mathbf{D}}}
\def\rmE{{\mathbf{E}}}
\def\rmF{{\mathbf{F}}}
\def\rmG{{\mathbf{G}}}
\def\rmH{{\mathbf{H}}}
\def\rmI{{\mathbf{I}}}
\def\rmJ{{\mathbf{J}}}
\def\rmK{{\mathbf{K}}}
\def\rmL{{\mathbf{L}}}
\def\rmM{{\mathbf{M}}}
\def\rmN{{\mathbf{N}}}
\def\rmO{{\mathbf{O}}}
\def\rmP{{\mathbf{P}}}
\def\rmQ{{\mathbf{Q}}}
\def\rmR{{\mathbf{R}}}
\def\rmS{{\mathbf{S}}}
\def\rmT{{\mathbf{T}}}
\def\rmU{{\mathbf{U}}}
\def\rmV{{\mathbf{V}}}
\def\rmW{{\mathbf{W}}}
\def\rmX{{\mathbf{X}}}
\def\rmY{{\mathbf{Y}}}
\def\rmZ{{\mathbf{Z}}}
% Elements of random matrices
\def\ermA{{\textnormal{A}}}
\def\ermB{{\textnormal{B}}}
\def\ermC{{\textnormal{C}}}
\def\ermD{{\textnormal{D}}}
\def\ermE{{\textnormal{E}}}
\def\ermF{{\textnormal{F}}}
\def\ermG{{\textnormal{G}}}
\def\ermH{{\textnormal{H}}}
\def\ermI{{\textnormal{I}}}
\def\ermJ{{\textnormal{J}}}
\def\ermK{{\textnormal{K}}}
\def\ermL{{\textnormal{L}}}
\def\ermM{{\textnormal{M}}}
\def\ermN{{\textnormal{N}}}
\def\ermO{{\textnormal{O}}}
\def\ermP{{\textnormal{P}}}
\def\ermQ{{\textnormal{Q}}}
\def\ermR{{\textnormal{R}}}
\def\ermS{{\textnormal{S}}}
\def\ermT{{\textnormal{T}}}
\def\ermU{{\textnormal{U}}}
\def\ermV{{\textnormal{V}}}
\def\ermW{{\textnormal{W}}}
\def\ermX{{\textnormal{X}}}
\def\ermY{{\textnormal{Y}}}
\def\ermZ{{\textnormal{Z}}}
% Vectors
\def\vzero{{\bm{0}}}
\def\vone{{\bm{1}}}
\def\vmu{{\bm{\mu}}}
\def\vtheta{{\bm{\theta}}}
\def\va{{\bm{a}}}
\def\vb{{\bm{b}}}
\def\vc{{\bm{c}}}
\def\vd{{\bm{d}}}
\def\ve{{\bm{e}}}
\def\vf{{\bm{f}}}
\def\vg{{\bm{g}}}
\def\vh{{\bm{h}}}
\def\vi{{\bm{i}}}
\def\vj{{\bm{j}}}
\def\vk{{\bm{k}}}
\def\vl{{\bm{l}}}
\def\vm{{\bm{m}}}
\def\vn{{\bm{n}}}
\def\vo{{\bm{o}}}
\def\vp{{\bm{p}}}
\def\vq{{\bm{q}}}
\def\vr{{\bm{r}}}
\def\vs{{\bm{s}}}
\def\vt{{\bm{t}}}
\def\vu{{\bm{u}}}
\def\vv{{\bm{v}}}
\def\vw{{\bm{w}}}
\def\vx{{\bm{x}}}
\def\vy{{\bm{y}}}
\def\vz{{\bm{z}}}
% Elements of vectors
\def\evalpha{{\alpha}}
\def\evbeta{{\beta}}
\def\evepsilon{{\epsilon}}
\def\evlambda{{\lambda}}
\def\evomega{{\omega}}
\def\evmu{{\mu}}
\def\evpsi{{\psi}}
\def\evsigma{{\sigma}}
\def\evtheta{{\theta}}
\def\eva{{a}}
\def\evb{{b}}
\def\evc{{c}}
\def\evd{{d}}
\def\eve{{e}}
\def\evf{{f}}
\def\evg{{g}}
\def\evh{{h}}
\def\evi{{i}}
\def\evj{{j}}
\def\evk{{k}}
\def\evl{{l}}
\def\evm{{m}}
\def\evn{{n}}
\def\evo{{o}}
\def\evp{{p}}
\def\evq{{q}}
\def\evr{{r}}
\def\evs{{s}}
\def\evt{{t}}
\def\evu{{u}}
\def\evv{{v}}
\def\evw{{w}}
\def\evx{{x}}
\def\evy{{y}}
\def\evz{{z}}
% Matrix
\def\mA{{\bm{A}}}
\def\mB{{\bm{B}}}
\def\mC{{\bm{C}}}
\def\mD{{\bm{D}}}
\def\mE{{\bm{E}}}
\def\mF{{\bm{F}}}
\def\mG{{\bm{G}}}
\def\mH{{\bm{H}}}
\def\mI{{\bm{I}}}
\def\mJ{{\bm{J}}}
\def\mK{{\bm{K}}}
\def\mL{{\bm{L}}}
\def\mM{{\bm{M}}}
\def\mN{{\bm{N}}}
\def\mO{{\bm{O}}}
\def\mP{{\bm{P}}}
\def\mQ{{\bm{Q}}}
\def\mR{{\bm{R}}}
\def\mS{{\bm{S}}}
\def\mT{{\bm{T}}}
\def\mU{{\bm{U}}}
\def\mV{{\bm{V}}}
\def\mW{{\bm{W}}}
\def\mX{{\bm{X}}}
\def\mY{{\bm{Y}}}
\def\mZ{{\bm{Z}}}
\def\mBeta{{\bm{\beta}}}
\def\mPhi{{\bm{\Phi}}}
\def\mLambda{{\bm{\Lambda}}}
\def\mSigma{{\bm{\Sigma}}}
% Tensor
\DeclareMathAlphabet{\mathsfit}{\encodingdefault}{\sfdefault}{m}{sl}
\SetMathAlphabet{\mathsfit}{bold}{\encodingdefault}{\sfdefault}{bx}{n}
\newcommand{\tens}[1]{\bm{\mathsfit{#1}}}
\def\tA{{\tens{A}}}
\def\tB{{\tens{B}}}
\def\tC{{\tens{C}}}
\def\tD{{\tens{D}}}
\def\tE{{\tens{E}}}
\def\tF{{\tens{F}}}
\def\tG{{\tens{G}}}
\def\tH{{\tens{H}}}
\def\tI{{\tens{I}}}
\def\tJ{{\tens{J}}}
\def\tK{{\tens{K}}}
\def\tL{{\tens{L}}}
\def\tM{{\tens{M}}}
\def\tN{{\tens{N}}}
\def\tO{{\tens{O}}}
\def\tP{{\tens{P}}}
\def\tQ{{\tens{Q}}}
\def\tR{{\tens{R}}}
\def\tS{{\tens{S}}}
\def\tT{{\tens{T}}}
\def\tU{{\tens{U}}}
\def\tV{{\tens{V}}}
\def\tW{{\tens{W}}}
\def\tX{{\tens{X}}}
\def\tY{{\tens{Y}}}
\def\tZ{{\tens{Z}}}
% Graph
\def\gA{{\mathcal{A}}}
\def\gB{{\mathcal{B}}}
\def\gC{{\mathcal{C}}}
\def\gD{{\mathcal{D}}}
\def\gE{{\mathcal{E}}}
\def\gF{{\mathcal{F}}}
\def\gG{{\mathcal{G}}}
\def\gH{{\mathcal{H}}}
\def\gI{{\mathcal{I}}}
\def\gJ{{\mathcal{J}}}
\def\gK{{\mathcal{K}}}
\def\gL{{\mathcal{L}}}
\def\gM{{\mathcal{M}}}
\def\gN{{\mathcal{N}}}
\def\gO{{\mathcal{O}}}
\def\gP{{\mathcal{P}}}
\def\gQ{{\mathcal{Q}}}
\def\gR{{\mathcal{R}}}
\def\gS{{\mathcal{S}}}
\def\gT{{\mathcal{T}}}
\def\gU{{\mathcal{U}}}
\def\gV{{\mathcal{V}}}
\def\gW{{\mathcal{W}}}
\def\gX{{\mathcal{X}}}
\def\gY{{\mathcal{Y}}}
\def\gZ{{\mathcal{Z}}}
% Sets
\def\sA{{\mathbb{A}}}
\def\sB{{\mathbb{B}}}
\def\sC{{\mathbb{C}}}
\def\sD{{\mathbb{D}}}
% Don't use a set called E, because this would be the same as our symbol
% for expectation.
\def\sF{{\mathbb{F}}}
\def\sG{{\mathbb{G}}}
\def\sH{{\mathbb{H}}}
\def\sI{{\mathbb{I}}}
\def\sJ{{\mathbb{J}}}
\def\sK{{\mathbb{K}}}
\def\sL{{\mathbb{L}}}
\def\sM{{\mathbb{M}}}
\def\sN{{\mathbb{N}}}
\def\sO{{\mathbb{O}}}
\def\sP{{\mathbb{P}}}
\def\sQ{{\mathbb{Q}}}
\def\sR{{\mathbb{R}}}
\def\sS{{\mathbb{S}}}
\def\sT{{\mathbb{T}}}
\def\sU{{\mathbb{U}}}
\def\sV{{\mathbb{V}}}
\def\sW{{\mathbb{W}}}
\def\sX{{\mathbb{X}}}
\def\sY{{\mathbb{Y}}}
\def\sZ{{\mathbb{Z}}}
% Entries of a matrix
\def\emLambda{{\Lambda}}
\def\emA{{A}}
\def\emB{{B}}
\def\emC{{C}}
\def\emD{{D}}
\def\emE{{E}}
\def\emF{{F}}
\def\emG{{G}}
\def\emH{{H}}
\def\emI{{I}}
\def\emJ{{J}}
\def\emK{{K}}
\def\emL{{L}}
\def\emM{{M}}
\def\emN{{N}}
\def\emO{{O}}
\def\emP{{P}}
\def\emQ{{Q}}
\def\emR{{R}}
\def\emS{{S}}
\def\emT{{T}}
\def\emU{{U}}
\def\emV{{V}}
\def\emW{{W}}
\def\emX{{X}}
\def\emY{{Y}}
\def\emZ{{Z}}
\def\emSigma{{\Sigma}}
% entries of a tensor
% Same font as tensor, without \bm wrapper
\newcommand{\etens}[1]{\mathsfit{#1}}
\def\etLambda{{\etens{\Lambda}}}
\def\etA{{\etens{A}}}
\def\etB{{\etens{B}}}
\def\etC{{\etens{C}}}
\def\etD{{\etens{D}}}
\def\etE{{\etens{E}}}
\def\etF{{\etens{F}}}
\def\etG{{\etens{G}}}
\def\etH{{\etens{H}}}
\def\etI{{\etens{I}}}
\def\etJ{{\etens{J}}}
\def\etK{{\etens{K}}}
\def\etL{{\etens{L}}}
\def\etM{{\etens{M}}}
\def\etN{{\etens{N}}}
\def\etO{{\etens{O}}}
\def\etP{{\etens{P}}}
\def\etQ{{\etens{Q}}}
\def\etR{{\etens{R}}}
\def\etS{{\etens{S}}}
\def\etT{{\etens{T}}}
\def\etU{{\etens{U}}}
\def\etV{{\etens{V}}}
\def\etW{{\etens{W}}}
\def\etX{{\etens{X}}}
\def\etY{{\etens{Y}}}
\def\etZ{{\etens{Z}}}
% The true underlying data generating distribution
\newcommand{\pdata}{p_{\rm{data}}}
% The empirical distribution defined by the training set
\newcommand{\ptrain}{\hat{p}_{\rm{data}}}
\newcommand{\Ptrain}{\hat{P}_{\rm{data}}}
% The model distribution
\newcommand{\pmodel}{p_{\rm{model}}}
\newcommand{\Pmodel}{P_{\rm{model}}}
\newcommand{\ptildemodel}{\tilde{p}_{\rm{model}}}
% Stochastic autoencoder distributions
\newcommand{\pencode}{p_{\rm{encoder}}}
\newcommand{\pdecode}{p_{\rm{decoder}}}
\newcommand{\precons}{p_{\rm{reconstruct}}}
\newcommand{\laplace}{\mathrm{Laplace}} % Laplace distribution
\newcommand{\E}{\mathbb{E}}
\newcommand{\Ls}{\mathcal{L}}
\newcommand{\R}{\mathbb{R}}
\newcommand{\emp}{\tilde{p}}
\newcommand{\lr}{\alpha}
\newcommand{\reg}{\lambda}
\newcommand{\rect}{\mathrm{rectifier}}
\newcommand{\softmax}{\mathrm{softmax}}
\newcommand{\sigmoid}{\sigma}
\newcommand{\softplus}{\zeta}
\newcommand{\KL}{D_{\mathrm{KL}}}
\newcommand{\Var}{\mathrm{Var}}
\newcommand{\standarderror}{\mathrm{SE}}
\newcommand{\Cov}{\mathrm{Cov}}
% Wolfram Mathworld says $L^2$ is for function spaces and $\ell^2$ is for vectors
% But then they seem to use $L^2$ for vectors throughout the site, and so does
% wikipedia.
\newcommand{\normlzero}{L^0}
\newcommand{\normlone}{L^1}
\newcommand{\normltwo}{L^2}
\newcommand{\normlp}{L^p}
\newcommand{\normmax}{L^\infty}
\newcommand{\parents}{Pa} % See usage in notation.tex. Chosen to match Daphne's book.
\DeclareMathOperator*{\argmax}{arg\,max}
\DeclareMathOperator*{\argmin}{arg\,min}
\DeclareMathOperator{\sign}{sign}
\DeclareMathOperator{\Tr}{Tr}
\let\ab\allowbreak
\section{Introduction}\label{sec.intro}
Graph data are pervasive due to their ability to model complex relationships between entities in various applications, including social network analysis \citep{fan2019graph,yu2023learning}, Web mining \citep{xu2022evidence,wang2023brave}, and content recommendation systems \citep{qu2023semi,zhang2024g}.
%citation graphs \cite{kanakia2019scalable,xiong2017explicit} and molecular structures \cite{yu2023learning,zaidi2022pre}.
In many of these applications, the graph structures evolve over time, such as users commenting on posts in a social network \citep{kumar2018community,iba2010analyzing}, editing pages in a crowdsourcing project like Wikipedia \citep{kumar2015vews}, or interacting with courses in an online learning platform \citep{liyanagunawardena2013moocs}. Such graphs are termed \emph{dynamic} graphs \citep{barros2021survey,skarding2021foundations}, in contrast to conventional \emph{static} graphs that maintain an unchanging structure.
On dynamic graphs, dynamic graph neural networks (DGNNs) \citep{pareja2020evolvegcn,xu2020inductive} have been widely applied. In a typical design for DGNNs, each node updates its representation by iteratively receiving and aggregating messages from its neighbors in a time-dependent manner.
While DGNNs are often pre-trained on the link prediction task, the downstream application could involve a different task, such as node classification, leading to a significant gap between pre-training and task objectives. More advanced pre-training and fine-tuning strategies on dynamic graphs \citep{bei2023cpdg,chen2022pre,tian2021self} also suffer from the same limitation. They first pre-train a model for dynamic graphs based on various task-agnostic self-supervised signals on the graph, and then update the pre-trained model weights in a fine-tuning phase based on task-specific labels pertinent to the downstream application. Likewise, the pre-training and downstream objectives can differ significantly, resulting in a notable gap between them.
To bridge the gap between pre-training and downstream objectives \citep{liu2023pre}, prompt learning has first emerged in language models \citep{brown2020language}. Fundamentally, a prompt serves to reformulate the input for the downstream task to align with the pre-trained model, while freezing the pre-trained weights. Given far fewer parameters in a prompt than the pre-trained model, prompt learning is parameter-efficient and effective especially in data-scarce scenarios \citep{yu2024few}. Inspired by the success of prompts in the language domain, % \citep{brown2020language},
recent studies have explored prompt learning on graphs \citep{liu2023graphprompt,sun2022gppt,sun2023all}. %,yu2023multigprompt}.
However, these studies only focus on static graphs, neglecting the unique challenges brought by dynamic graphs.
In this work, we explore \textbf{Prompt} learning for \textbf{Dy}namic \textbf{G}raphs, and propose a framework called \model. The problem is non-trivial due to two key challenges.
\begin{figure}[t]
\centering
\includegraphics[width=1\linewidth]{figures/intro.pdf}
\vspace{-5mm}
\caption{Motivation of \model. (a) Users comment on different topics over time. (b) Evolving node-time patterns in Wikipedia, as node and time mutually characterize each other.}
\label{fig.intro-motivation}
\end{figure}
First, how do we design prompts to \textit{bridge temporal variations across time, in addition to divergent task objectives?} Generally, both task objectives and the timing of pre-training and downstream tasks can be inconsistent. For example, consider a user discussion network such as Reddit.com, as illustrated in Fig.~\ref{fig.intro-motivation}(a), where users and posts are represented as nodes, and an edge is formed when a user comments on a post. On this dynamic graph, a typical pre-training task involves link prediction---predicting whether a user comments on (i.e., links to) a video game post at time $t_1$. In contrast, the downstream task could be the status classification of the user's account, e.g., determining whether it has been suspended at time $t_2$.
While previous work on static graphs \citep{liu2023graphprompt,sun2022gppt,sun2023all} employs prompts to narrow the task objective gap, the temporal gap remains unsolved.
In particular, even when the downstream task also involves link prediction, the evolving graph structure introduces a temporal gap between the pre-training and downstream phases.
In \model, we introduce dual prompts, consisting of a \emph{node prompt} and a \emph{time prompt}. On the one hand, a \textit{node prompt} alters the node features to reformulate task input and bridge the task gap, similar to previous work \citep{liu2023graphprompt,fang2022universal}. On the other hand, a \textit{time prompt} adjusts the time features, capturing the temporal evolution of the dynamic graph, thereby narrowing inconsistencies that arise from varying priorities at different times.
Second, how do we capture \textit{evolving patterns across different nodes and time points, driven by the dynamic interplay between them?}
Different nodes exhibit unique timelines, evolving distinctively over time as nodes and time mutually influence each other. For example, in the user discussion network, a student may comment on open source code in the morning, while commenting on video games in the evening. However, as shown in Fig.~\ref{fig.intro-motivation}(a), a professor may comment on academic policies at noon, while discussing family topics at night.
Therefore, node and time patterns mutually characterize each other:
the behavior of the same node may be influenced by different times, while the behavior of a node at a specific time may be influenced by the characteristics of the node. Hence, the evolving node-time patterns, as shown in Fig.~\ref{fig.intro-motivation}(b), may differ from the patterns observed in the pre-training data. However, previous studies neglect the fine-grained interplay between the node and time patterns downstream.
Note that some works \citep{tan2023virtual,wen2023prompt} have designed individual prompts or tokens tailored to each node,
and a contemporary approach \citep{chen2024prompt} has developed time-aware prompts. However, they do not account for the mutual characterization between nodes and time, failing to capture the fine-grained, evolving node-time patterns.
In \model, inspired by conditional prompt learning \citep{zhou2022conditional}, we propose a series of node and time prompts generated by \emph{dual condition-nets}. Specifically, a \emph{time condition-net} generates a sequence of time-conditioned node prompts for each node.
These time-conditioned node prompts reflect temporal subtleties at different times for the same node, thereby better aligning the node's pattern at different times with those observed during pre-training compared to using a fixed node prompt across all time points.
Likewise, a \emph{node condition-net} generates a sequence of node-conditioned time prompts at each timestamp.
The node-conditioned time prompts account for node characteristics, thereby better aligning the time patterns of different nodes with those captured by the pre-trained model compared to using a fixed time prompt for all nodes.
It is worth noting that the condition-nets generate prompts conditioned on the input features rather than directly parameterizing the prompts, significantly reducing the number of learnable parameters in the downstream prompt-tuning phase.
In summary, the contributions of this work are threefold.
(1) We introduce \model\ for dynamic graph modeling, proposing dual prompts to narrow the gaps arising from both task and dynamic differences between pre-training and downstream phases.
(2) We recognize that node and time patterns mutually characterize each other and further propose dual condition-nets to generate a series of node and time prompts, thereby adapting to downstream node-time patterns.
(3) We conduct extensive experiments on four benchmark datasets, demonstrating the superior performance of \model\ in comparison to state-of-the-art approaches.
\section{Related Work}
\stitle{Dynamic graph learning.}
Real-world graph structures often evolve over time. To model such dynamic evolution, various \textit{continuous-time} dynamic graph learning techniques have been proposed. Typically, these methods update node representations by iteratively receiving and aggregating messages from their neighbors in a time-dependent manner \citep{skarding2021foundations,duan2024cat}.
Some approaches employ dynamic random walks to depict structural changes \citep{nguyen2018continuous,wang2021inductive}, while others incorporate a time encoder to integrate temporal context with structural modeling \citep{xu2020inductive,cong2022we,rossi2020temporal,yu2023towards}. Furthermore, some researchers utilize temporal point processes to model structural evolution or node communications \citep{kumar2019predicting,trivedi2019dyrep,wen2022trend}.
However, these methods are typically trained on link prediction tasks, whereas downstream applications may involve different tasks, such as node classification, creating a significant gap between training and task objectives. This gap impedes effective knowledge transfer, adversely impacting downstream task performance.
\stitle{Graph pre-training.}
More advanced dynamic graph pre-training methods also face similar problems. Extending from graph pre-training methods \citep{hu2019strategies,hu2020gpt,jiang2023incomplete}, dynamic graph pre-training captures inherent properties of dynamic graphs using various task-agnostic, self-supervised signals through strategies such as structural and temporal contrastive learning \citep{bei2023cpdg,tian2021self,li2022mining}, dynamic graph generation \citep{chen2022pre}, and curvature-varying Riemannian graph neural networks \citep{sun2022self}. They then attempt to transfer the prior knowledge to downstream tasks through fine-tuning with task-specific supervision.
Nonetheless, a gap persists between the objectives of pre-training and fine-tuning \citep{liu2023pre,yu2023generalized}. While pre-training seeks to extract fundamental insights from graphs without supervision, fine-tuning is tailored to specific supervision for a given downstream task.
\stitle{Graph prompt learning.}
First emerging in the language domain, prompt learning effectively bridges the gap between pre-training and downstream objectives \citep{brown2020language}. With a unified template, prompts are specifically tailored for each downstream task, aligning it more closely with the pre-trained model while freezing the pre-trained weights. Prompt learning has been widely adopted on static graphs \citep{liu2023graphprompt,sun2023all,fang2022universal,tan2023virtual,yu2023generalized,yu2024non,yu2024text,yu2025samgpt,yu2025gcot,yu2023hgprompt}, yet its application in dynamic graphs remains an open question. A contemporary study, TIGPrompt \citep{chen2024prompt}, proposes a prompt generator to learn time-aware node prompts. However, it only considers the temporal factor in node features, overlooking that time prompts for each node can also be influenced by node features.
\section{Preliminaries}
In this section, we present related preliminaries and introduce our scope.
\stitle{Dynamic graph.}
Dynamic graphs are categorized into \textit{discrete-time} dynamic graphs and \textit{continuous-time} dynamic graphs \citep{skarding2021foundations}. The former consists of a series of static graph snapshots, whereas the latter is depicted as a sequence of events on a continuous timeline.
In this work, we adopt the more general continuous-time definition, where a \textit{dynamic graph} is represented by \( G = (V, E, T) \). Here, $V$ denotes the set of nodes, $E$ denotes the set of edges, and $T$ represents the time domain. In particular, each edge $(v_i,v_j, t)\in E$ indicates a specific interaction (or event) between nodes $v_i$ and $v_j$ at time $t$. Each node is associated with a temporal feature vector \( \vec{x}_{t,v} \in \mathbb{R}^d \) evolving over time, represented as a row in the temporal feature matrix $\mathbf{X}_t \in \mathbb{R}^{|V| \times d}$.
\stitle{Dynamic graph neural network.}
A popular backbone for static graphs is message-passing graph neural networks (GNNs) \citep{wu2020comprehensive}. Formally, in the $l$-th GNN layer, the embedding of node $v$, represented as $\vec{h}^l_v$, is calculated based on the embeddings from the preceding layer:
\begin{equation}
\vec{h}^l_v = \mathtt{Aggr}(\vec{h}^{l-1}_v, \{\vec{h}^{l-1}_u : u\in\bN_v\}), %; \theta^l),
\end{equation}
where $\bN_v$ denotes the set of neighboring nodes of $v$,
and $\mathtt{Aggr}(\cdot)$ is a neighborhood aggregation function.
Extending to dynamic graph neural networks (DGNNs), researchers integrate temporal contexts into neighborhood aggregation \citep{pareja2020evolvegcn,xu2020inductive}, often utilizing a \emph{time encoder} ($\mathtt{TE}$) to map timestamps or intervals \citep{cong2022we,rossi2020temporal} within a \emph{dynamic graph encoder} ($\mathtt{DGE}$). Consequently, in the $l$-th layer of a dynamic graph encoder, the embedding of node $v$ at time $t$, denoted by $\vec{h}^l_{t,v}$, is derived as
\begin{align}
\vec{h}^l_{t,v} = \mathtt{Aggr}(\mathtt{Fuse}(\vec{h}^{l-1}_{t,v},\mathtt{TE}(t)), \{\mathtt{Fuse}(\vec{h}^{l-1}_{t',u},\mathtt{TE}(t')) : (u,t')\in\bN_v\}), %; \theta^l),
\end{align}
where $\bN_v$ contains the historical neighbors of $v$, with $(u,t')\in \bN_v$ indicating that $u$ interacts with $v$ at time $t'<t$.
$\mathtt{Fuse}(\cdot)$ is a fusion operation that integrates structural and temporal contexts, such as concatenation \citep{rossi2020temporal} or addition \citep{trivedi2019dyrep}. For simplicity, the output node embedding from the final layer is denoted as $\vec{h}_{t,v}$, which is then utilized in a loss function for optimization.
Note that the time encoder maps the time domain to a vector space, $\mathtt{TE} : T \rightarrow \mathbb{R}^{d}$. A common implementation is
$\mathtt{TE}(t) =\frac{1}{\sqrt{d}} [ \cos(\omega_1 t), \sin(\omega_1 t), \ldots, \cos(\omega_{d/2} t), \sin(\omega_{d/2} t) ] ,$
where $\{\omega_1,\ldots,\omega_{d/2}\}$ is a set of learnable parameters \citep{xu2020inductive,cong2022we}.
\stitle{Scope of work.}
In this work, we pre-train a DGNN with a time encoder on dynamic graphs, employing the task of temporal link prediction \citep{wang2021inductive}.
For downstream tasks, we target two popular dynamic graph-based tasks, namely, temporal node classification and temporal link prediction\footnote{As dynamic graphs focus on the evolving structures within a graph, graph classification is rarely evaluated as a task \citep{skarding2021foundations,pareja2020evolvegcn,xu2020inductive,chen2024prompt}.}.
Specifically, we focus on \emph{data-scarce scenarios} with only limited data for downstream adaptation, as labeled data are often difficult or costly to obtain for node classification \citep{zhou2019meta,yao2020graph}, while nodes with sparse interactions are common in real-world link prediction tasks \citep{lee2019melu,pan2019warm}.
% In temporal node classification, % on a dynamic graph $G=(V, E, T)$ ,
% since node properties (e.g., account status in a social network) can vary as the graph evolves,
% the goal is to predict the class of a given node at a specific time $t$ from a predefined set of classes $Y$.
% In temporal link prediction, our goal is to predict whether a link exists between two nodes at a specific time in both transductive and inductive settings. The transductive setting involves nodes that have appeared during both pre-training and downstream prompt tuning, whereas the inductive setting exclusively predicts the existence of edges between nodes that have not been seen during both pre-training and downstream tuning.
\section{Proposed Approach}
In this section, we present our proposed model \model, starting with its overall framework, followed by its key components.
\subsection{Overall framework}
We illustrate the overall framework of \model\ in Fig.~\ref{fig.framework}, which consists of two phases: pre-training and downstream adaptation.
First, given a dynamic graph in Fig.~\ref{fig.framework}(a), we pre-train a DGNN based on temporal link prediction, as shown in Fig.~\ref{fig.framework}(b).
In particular, we employ a universal task template based on similarity calculation \citep{liu2023graphprompt}, which unifies different graph-based tasks, such as link prediction and node classification, across the pre-training and downstream phases.
Second, given a pre-trained DGNN, to enhance its adaptation to downstream tasks, we propose \textit{dual prompts}, as shown in Fig.~\ref{fig.framework}(c). The dual prompts comprise a \textit{node prompt} and a \textit{time prompt}, aiming to bridge the gaps caused by different task objectives and dynamic variations, respectively. Moreover, extending the dual prompts, we propose \emph{dual condition-nets} to generate a series of time-conditioned node prompts and node-conditioned time prompts. These conditional prompts are designed to capture the evolving patterns across nodes and time points, along with their mutual characterization, in a fine-grained and parameter-efficient manner.
\begin{figure*}[t]
\centering
\includegraphics[width=1\linewidth]{figures/framework.pdf}
\vspace{-6mm}
\caption{Overall framework of \model.}
\label{fig.framework}
\end{figure*}
\subsection{Pre-training}\label{sec.model.pre-train}
Pre-training techniques \citep{le2020contrastive} have been widely explored on dynamic graphs \citep{bei2023cpdg,chen2022pre,tian2021self}.
In particular, the most common pre-training task is temporal link prediction \citep{xu2020inductive,rossi2020temporal}, where links are readily available without the need for manual annotations.
Following these prior works, we adopt link prediction for pre-training, as illustrated in Fig.~\ref{fig.intro-motivation}(b). Its simplicity underscores the effectiveness and robustness of our prompt-based learning in downstream tasks.
Specifically, consider an event $(v,a,t)$, i.e., an interaction between nodes $v$ and $a$ at time $t$. Correspondingly, we construct a tuple $(v,a,b,t)$ such that nodes $v$ and $b$ are not linked at $t$, serving as a contrastive signal. For each tuple, the pre-training objective is to maximize the similarity between nodes $v$ and $a$ while minimizing that between $v$ and $b$. Similar to previous work \citep{you2020graph,yu2023generalized}, we optimize the following pre-training loss over pre-training data, $\mathcal{D}_\text{pre}$.
\begin{align} \label{eq.pre-train-loss}
\textstyle \bL_{\text{pre}}(\mathcal{D}_\text{pre};\Theta)= -\sum_{(v,a,b,t)\in\mathcal{D}_\text{pre}}\ln\frac{\exp\left(\frac{1}{\tau}\text{sim}(\vec{h}_{t,v},\vec{h}_{t,a})\right)}{\exp\left(\frac{1}{\tau}\text{sim}(\vec{h}_{t,v},\vec{h}_{t,b})\right)}.
\end{align}
In Eq.~(\ref{eq.pre-train-loss}), $\vec{h}_{t,v},\vec{h}_{t,a},\vec{h}_{t,b}$ represent the node embeddings of $v,a,b$ at time $t$, respectively. $\tau>0$ is a temperature hyperparameter. $\Theta$ denotes the set of learnable parameters of the DGNN, including those of the dynamic graph encoder and the time encoder.
The pre-trained parameters, $\Theta_\text{pre}=\arg\min_\Theta \mathcal{L}_{\text{pre}}(\mathcal{D}_\text{pre};\Theta)$, will be used to initialize the downstream models.
\subsection{Dual prompts}\label{sec.model.prompt}
%We employ LP within a specific timing as the pre-training task, while downstream tasks could be NC and LP in a different timing.
To reconcile task and temporal variations across pre-training and downstream tasks, we propose \textit{dual prompts}, comprising a node prompt and a time prompt, as shown in Fig.~\ref{fig.framework}(c).
\stitle{Node prompt.}
In language models, a prompt is a purposely designed textual description that reformulates the input for a downstream task, aligning the task with the pre-trained model \citep{brown2020language,jia2021scaling}. Likewise, a prompt for static graphs adjusts downstream node features \citep{liu2023graphprompt,fang2022universal}.
%Inspired by early graph prompt learning methods \cite{liu2023graphprompt,yu2023generalized},
%we design a learnable prompt to modify the input node features, guiding different downstream tasks leverage distinct aspects of nodes features based on their specific needs.
In line with previous graph prompt learning methods on static graphs \citep{liu2023graphprompt,yu2023generalized,yu2023multigprompt, yu2024non}, for each downstream task, we define a learnable vector $\vec{p}^\text{node}$ as the node prompt to modify the node features at a given time via element-wise multiplication. Concretely, the input features of node $v$ at time $t$, denoted by $\vec{x}_{t,v}$, are modified as:
\begin{align}\label{eq.prompt-stru}
\vec{x}^\text{node}_{t,v}=\vec{p}^\text{node}\odot\vec{x}_{t,v},
\end{align}
where $\odot$ is element-wise multiplication. The node prompt $\vec{p}^\text{node}$ has the same dimension as node features, altering the importance of various node feature dimensions.
\stitle{Time prompt.}
We adopt a time encoder $\mathtt{TE}$ to map the continuously valued time $t$ to time features, $\vec{f}_t=\mathtt{TE}(t)$.
%Continuous-time dynamic graph learning approaches generally employ a time encoder to map time $t$ to time features $\vec{f}_t$:
%\begin{align}\label{eq.time-encoder}
%\vec{f}_t=\mathtt{TE}(t),
%\end{align}
%where $\mathtt{TE}$ is the time encoder.
However, $\textsc{TE}$ is optimized based on the time periods in the pre-training data, which can be distinct from those in downstream tasks. To overcome the temporal gap, we propose a time prompt to adjust time features. Formally, for each downstream task, we define a learnable vector $\vec{p}^\text{time}$ as the \emph{time prompt}. Then, the time feature at $t$, denoted by $\vec{f}_t$, is reformulated as follows.
\begin{align}\label{eq.prompt-temp}
\vec{f}^\text{time}_t=\vec{p}^\text{time}\odot\vec{f}_t.
\end{align}
%Therefore, the time prompt reweighs the important of various time feature dimensions to align the downstream temporal patterns with the pre-trained model.
\subsection{Dual condition-nets}\label{sec.model.condition}
In dynamic graphs, the same node may exhibit distinct behaviors at different times. Meanwhile, different nodes may evolve differently over the same time interval. In other words, nodes and their temporal patterns influence and characterize each other over time, revealing fine-grained node-time patterns. To capture such interplay,
we turn to conditional prompt learning \citep{zhou2022conditional}.
Instead of learning a static prompt tailored to a specific task over the entire time span, we design \textit{dual condition-nets}, comprising a time condition-net that generates a sequence of time-conditioned node prompts for each node and a node condition-net that generates a sequence of node-conditioned time prompts at each timestamp, as shown in Fig.~\ref{fig.framework}(c).
In particular, each condition-net utilizes a lightweight network to generate prompts conditioned on the input features, enabling a more parameter-efficient approach to prompt tuning compared to directly parameterizing the prompts for each node and timestamp.
\stitle{Time condition-net.}
To incorporate time characteristics into node prompts, a straightforward way is to employ a series of learnable vectors as prompts for each distinct timestamp. However, this approach does not scale to large time intervals with many timestamps, as it quickly increases the number of tunable parameters.
Therefore, we propose a \textit{time condition-net}, which generates a series of \textit{time-conditioned node prompts}. Formally, the node prompt conditioned on time $t$ is given by
\begin{align}\label{eq.time-condition}
\tilde{\vec{p}}^\text{node}_t=\mathtt{TCN}(\vec{f}^\text{time}_t;\kappa),
\end{align}
where $\mathtt{TCN}$ is the time condition-net parameterized by $\kappa$.
This can be viewed as a form of hypernetwork \citep{ha2022hypernetworks}, where our condition-net $\mathtt{TCN}$ serves as a secondary network to generate the node prompts conditioned on the input time features. This formulation enables parameter-efficient prompt generation, especially when using a lightweight condition-net.
In our implementation, we opt for a simple multi-layer perceptron (MLP) with an efficient bottleneck structure \citep{wu2018reducing}. Therefore, for a series of events occurring at times $\{t_0, t_1, \ldots\}$, the time condition-net produces their corresponding time-conditioned node prompts $\{\tilde{\vec{p}}^\text{node}_{t_0}, \tilde{\vec{p}}^\text{node}_{t_1}, \ldots\}$. Note that these conditional prompts match the dimensionality of the node features. Then, the features of node $v$ at time $t$, which have been modified by the time prompt in Sect.~\ref{sec.model.prompt}, are further updated as
\begin{align}\label{eq.conditional-prompt-str}
\tilde{\vec{x}}^\text{node}_{t,v}=\tilde{\vec{p}}^\text{node}_t\odot\vec{x}^\text{node}_{t,v}.
\end{align}
\stitle{Node condition-net.}
Similarly, to incorporate node characteristics into time prompts, we propose a \textit{node condition-net} to generate a series of \emph{node-conditioned time prompts}, mirroring the time condition-net.
%Note that the feature condition-net employ the same framework as the time condition-net.
%Formally, %given an event $(v,u, t)$, where node $v$ links to node $u$ at time $t$,
The time prompt conditioned on the node features of $v$ at time $t$ is given by
\begin{align}\label{eq.node-condition}
\tilde{\vec{p}}^\text{time}_{t,v}=\mathtt{NCN}(\vec{x}^\text{node}_{t,v};\phi),
\end{align}
where $\mathtt{NCN}$ is the node condition-net parameterized by $\phi$, which is also implemented as an MLP with a bottleneck structure. % and $\kappa_\text{fea}$ denotes its learnable parameters.
Consequently, at time $t$, a series of node-conditioned time prompts $\{\tilde{\vec{p}}^\text{time}_{t,v_0},\tilde{\vec{p}}^\text{time}_{t,v_1},\ldots\}$ is generated for all nodes, which have the same dimension as the time features. These node-specific time prompts are then applied to each node at time $t$, as follows.
\begin{align}\label{eq.conditional-prompt-tem}
\tilde{\vec{f}}^{\text{time}}_{t,v}=\tilde{\vec{p}}^\text{time}_{t,v}\odot\vec{f}^\text{time}_t.
\end{align}
%Note that the feature-conditioned time prompts $\vec{cp}^\text{fea}_t$ have the same dimension as the time feature, thus customizing the importance of time feature conditioned on nodes features, further enhancing the alignment of downstream temporal patterns with pre-training phase.
\subsection{Prompt tuning}\label{sec.model.tuning}
Finally, the features adjusted by the prompts are fed into the dynamic graph encoder, $\mathtt{DGE}$. The embedding of node $v$ at time $t$ is calculated as follows:
\begin{align}\label{eq.dynamic-graph-encoder}
\vec{h}_{t,v} = \mathtt{DGE}\left(\mathtt{Fuse}(\tilde{\vec{x}}^{\text{node}}_{t,v}, \tilde{\vec{f}}^{\text{time}}_{t,v}), \left\{\mathtt{Fuse}(\tilde{\vec{x}}^{\text{node}}_{t',u}, \tilde{\vec{f}}^{\text{time}}_{t',v}) : (u,t')\in\bN_v\ \right\} \right).
\end{align}
To tune the prompts and condition-nets for a downstream task, we adopt similarity calculation as the task template \citep{liu2023graphprompt}, consistent with the pre-training loss. Specifically, for temporal node classification, consider a labeled set $\mathcal{D}_\text{down}=\{(v_1,y_1,t_1),(v_2,y_2,t_2),\ldots\}$, where each $v_i$ denotes a node, and $y_i\in Y$ is the class label of $v_i$ at time $t_i$. Then, the downstream loss is
\begin{align}\label{eq.loss}
\textstyle
\bL_{\text{down}}(\mathcal{D}_\text{down};\vec{p}^\text{node},\vec{p}^\text{time},\kappa,\phi) = -\sum_{(v_i,y_i,t_i)\in \mathcal{D}_\text{down}}\ln\frac{\exp\left(\frac{1}{\tau}\text{sim}(\vec{h}_{t_i,v_i},\bar{\vec{h}}_{t_i,y_i})\right)}{\sum_{y\in Y}\exp\left(\frac{1}{\tau}\text{sim}(\vec{h}_{t_i,v_i},\bar{\vec{h}}_{t_i,y})\right)},
\end{align}
where $\bar{{\vec{h}}}_{t_i,y}$ represents class $y$'s prototype embeddings \citep{liu2023graphprompt} at time $t_i$, which is obtained by the mean embeddings of examples in class $y$ at time $t_i$.
For temporal link prediction, we utilize the same loss as the pre-training loss in Eq.~(\ref{eq.pre-train-loss}).
During prompt tuning, we only update the lightweight dual prompt vectors ($\vec{p}^\text{node},\vec{p}^\text{time}$) and the parameters of the dual condition-nets ($\kappa,\phi$), whilst freezing the pre-trained DGNN weights. This parameter-efficient tuning approach is amenable to label-scarce settings, where $\mathcal{D}_\text{down}$ comprises only a limited number of training examples.
%\subsection{Algorithm and Complexity Analysis}
We outline the key steps for prompt tuning in Algorithm~\ref{alg.prompt} in Appendix~\ref{app.alg} and assess its complexity in Appendix~\ref{complexity}.
\section{Experiments}
In this section, we conduct experiments to evaluate \model\ and analyze the empirical results.
\subsection{Experimental setup}\label{sec:expt:setup}
\stitle{Datasets.}
We utilize four benchmark datasets for evaluation: \textit{Wikipedia}, \textit{Reddit}, \textit{MOOC} and \textit{Genre}. We provide further details of the datasets in Appendix~\ref{app.dataset}.
% (1) \textit{Wikipedia} and (2) \textit{Reddit} are both user--post networks. Each dataset represents a monthly record, with nodes representing users or posts. In \textit{Wikipedia}, an edge denotes a user editing a post, while in \textit{Reddit}, an edge represents a user commenting on a post. Dynamic class labels indicate whether a user's account has been temporarily suspended.
% (3) \textit{MOOC} is a user--course network, where nodes represent users and courses, and edges represent actions by users on the courses. Dynamic class labels indicate whether the user has dropped out of the course following this action.
% (4) \textit{Genre} is a user--music genre network, where a node corresponds to a user or genre, and an edge represents a user listening to a particular genre at a specific moment in time. Dynamic class labels reflect users' preferred music genre.
% We provide further details of the datasets in Appendix~\ref{app.dataset}.
% \begin{table}[tbp]
% \centering
% \caption{Summary of datasets.}
% \label{table.dataset}
% \footnotesize
% \resizebox{1\linewidth}{!}{%
% \begin{tabular}{@{}lcccccc@{}}
% \toprule
% \multirow{2}*{Dataset} & Nodes & Edges & Node & Dynamic & Feature & Time\\
% & num &num &classes &labels &dimension &span\\\midrule
% Wikipedia & 9,227 & 157,474 & 2 & 217 & 172 & 30 days\\
% Reddit & 11,000 & 672,447 & 2 & 366 & 172 & 30 days\\
% MOOC & 7,144 & 411,749 & 2 & 4,066 & 172 & 30 days\\
% Genre & & & & & &\\
% \bottomrule
% \end{tabular}}
% \end{table}
\stitle{Downstream tasks.}\label{sec.implementation}
We conduct temporal node classification and temporal link prediction to evaluate the performance of \model. For link prediction, we perform experiments in both transductive and inductive settings, depending on whether nodes in the testing set have appeared in the pre-training or downstream tuning data. %Moreover, to evaluate the performance of \model~under varying levels of data availability, we conduct experiments in both data-scarce and data-abundant scenarios.
In this study, we investigate the pre-training and prompt learning on dynamic graphs, which differs from previous DGNNs \citep{rossi2020temporal,xu2020inductive}.
Therefore, we adopt a new data split to accommodate the pre-training and downstream tuning steps.
Specifically, given a \emph{chronologically ordered} sequence of events (i.e., edges with timestamps), we use the first 80\% for pre-training. Note that we pre-train a DGNN only once for each dataset and subsequently employ the same pre-trained model for all downstream tasks. The remaining 20\% of the events are used for downstream tasks. Specifically, they are further split into 1\%/1\%/18\% subsets, where the first 1\% is reserved as the training pool for downstream prompt tuning, and the next 1\% as the validation pool, with the last 18\% for downstream testing.
% For the data-scarce scenario,
To construct a downstream task,
we start by sampling 30 events (about 0.01\% of the whole dataset) from the training pool, ensuring that there is at least one user per class. Then, for node classification, we take the user nodes as training instances for downstream adaptation, and their ground-truth labels are given by their labels at the time of the corresponding event.
For link prediction, the downstream training pool also helps with adaptation due to its temporal proximity to the test set. Specifically, we take the sampled events as positive instances, while further sampling one negative instance for each positive instance. Specifically, for each event $(v,a,t)$, we sample one negative instance $(v,b,t)$ such that $b$ is a node taken from the training pool that is not linked to $v$ at time $t$. The validation sets are constructed by repeating the same process on the validation pool.
Finally, the test set for node classification simply comprises all user nodes in the testing events, while the test set for link prediction is expanded to include negative instances following a similar sampling process as in training. Furthermore, for inductive link prediction, we remove instances from the test set if their nodes have appeared in the pre-training or downstream training data.
We repeat this sampling process for 100 times to construct 100 tasks for node classification and link prediction.
% In contrast, for the data-abundant scenario, we use the entire 1\% training and validation pool for downstream adaptation, while the test set remains the same.
\stitle{Evaluation.} To evaluate the performance on the tasks, we employ AUC-ROC for both node classification \citep{xu2020inductive,rossi2020temporal} and link prediction \citep{sun2022self,bei2023cpdg}, following prior work. For each of the 100 tasks, we repeat the experiments with five different random seeds, and report the average and standard deviation over the 500 results. %\textcolor{blue}{for data-scarce setting and 5 results for data-abundant setting.}
\begin{table*}[tbp]
\centering
\caption{AUC-ROC (\%) evaluation of temporal node classification and link prediction.}
\label{table.fewshot}
\addtolength{\tabcolsep}{-1mm}
\resizebox{\textwidth}{!}{%
\begin{tabular}{@{}l|cccc|cccc|cccc@{}}
\toprule
\multirow{2}*{Methods} & \multicolumn{4}{c|}{{Node Classification}} & \multicolumn{4}{c|}{{Transductive Link Prediction}} & \multicolumn{4}{c}{{Inductive Link Prediction}} \\
\multicolumn{1}{c|}{} & {Wikipedia} & {Reddit} & {MOOC} & {Genre} & {Wikipedia} & {Reddit} & {MOOC} & {Genre} & {Wikipedia} & {Reddit} & {MOOC} & {Genre} \\ \midrule\midrule
% \method{GCN} &58.86 &48.25 &49.93 &49.61&50.01 &49.82 &49.60 & 49.90& 49.16 \\
% \method{GAT} &62.81&47.95 & 50.01&52.34 &50.04 &55.74 &52.29&49.85 &54.01 \\\midrule
\method{GCN-ROLAND} &58.86\text{\scriptsize ±10.3} &48.25\text{\scriptsize ±9.57} &49.93\text{\scriptsize ±6.74} &46.33\text{\scriptsize ±3.97} &49.61\text{\scriptsize ±3.12} &50.01\text{\scriptsize ±2.53} &49.82\text{\scriptsize ±1.44} &49.15\text{\scriptsize ±3.74} &49.60\text{\scriptsize ±2.37} &49.90\text{\scriptsize ±1.64} &49.16\text{\scriptsize ±2.48} &47.25\text{\scriptsize ±2.97} \\
\method{GAT-ROLAND} &62.81\text{\scriptsize ±9.88} &47.95\text{\scriptsize ±8.42} &50.01\text{\scriptsize ±6.34} & 47.26\text{\scriptsize ±3.49}&52.34\text{\scriptsize ±1.82} &50.04\text{\scriptsize ±1.98} &55.74\text{\scriptsize ±3.71} &47.69\text{\scriptsize ±2.81} &52.29\text{\scriptsize ±1.97} &49.85\text{\scriptsize ±2.35} &54.01\text{\scriptsize ±2.16} &49.38\text{\scriptsize ±2.72}\\%\midrule
% \method{DyRep} &48.82\text{\scriptsize ±13.37} &52.38\text{\scriptsize ±6.90} &45.96\text{\scriptsize ±5.07} &50.48\text{\scriptsize ±1.77} &58.45\text{\scriptsize ±0.12} &58.02\text{\scriptsize ±0.31} &50.29\text{\scriptsize ±0.13} &88.89\text{\scriptsize ±2.23} &56.87\text{\scriptsize ±0.18} &57.25\text{\scriptsize ±0.28} &50.34\text{\scriptsize ±0.16} & 87.38\text{\scriptsize ±2.43} \\
% \method{JODIE} &51.37\text{\scriptsize ±14.38} &49.80\text{\scriptsize ±6.87} &50.53\text{\scriptsize ±5.47} &50.34\text{\scriptsize ±2.13} &62.40\text{\scriptsize ±3.43} &59.81\text{\scriptsize ±7.59} &50.71\text{\scriptsize ±3.83} &91.19\text{\scriptsize ±3.27} &59.59\text{\scriptsize ±3.28} &61.28\text{\scriptsize ±6.15} &50.57\text{\scriptsize ±3.97} &89.79\text{\scriptsize ±3.16} \\
\method{TGAT} &67.00\text{\scriptsize ±5.35} &53.64\text{\scriptsize ±5.50} &59.27\text{\scriptsize ±4.43} &51.26\text{\scriptsize ±2.31}&55.78\text{\scriptsize ±2.03} &62.43\text{\scriptsize ±1.86} &51.49\text{\scriptsize ±1.30} &69.11\text{\scriptsize ±3.89} &48.21\text{\scriptsize ±1.55} &57.30\text{\scriptsize ±0.70} &51.42\text{\scriptsize ±4.27} &48.38\text{\scriptsize ±4.72}\\
\method{TGN} &50.61\text{\scriptsize ±13.6} &49.54\text{\scriptsize ±6.23} &50.33\text{\scriptsize ±4.47} & 50.72\text{\scriptsize ±2.31}&72.48\text{\scriptsize ±0.19} &67.37\text{\scriptsize ±0.07} &54.60\text{\scriptsize ±0.80} &\text{86.46}\text{\scriptsize ±2.84} &74.38\text{\scriptsize ±0.29} &69.81\text{\scriptsize ±0.08} &54.62\text{\scriptsize ±0.72} &87.17\text{\scriptsize ±2.68}\\
\method{TREND} &69.92\text{\scriptsize ±9.27} &\text{64.85\scriptsize ±4.71} &66.79\text{\scriptsize ±5.44} & 50.34\text{\scriptsize ±1.62}&63.24\text{\scriptsize ±0.71}&80.42\text{\scriptsize ±0.45}&58.70\text{\scriptsize ±0.78} & 52.78\text{\scriptsize ±1.14}
&50.15\text{\scriptsize ±0.90}&65.13\text{\scriptsize ±0.54}&57.52\text{\scriptsize ±1.01} &45.31\text{\scriptsize ±0.43} \\
\method{GraphMixer} & 65.43\text{\scriptsize ±4.21} & 60.21\text{\scriptsize ±5.36} & 63.72\text{\scriptsize ±4.98}& 50.15\text{\scriptsize ±1.49}& 59.73\text{\scriptsize ±0.35} & 61.88\text{\scriptsize ±0.11} & 52.42\text{\scriptsize ±1.38}&60.83\text{\scriptsize ±3.25} & 51.34\text{\scriptsize ±0.84} & 57.64\text{\scriptsize ±0.31}&51.16\text{\scriptsize ±2.59} &56.32\text{\scriptsize ±3.08}\\
\midrule
\method{DDGCL} &65.15\text{\scriptsize ±4.54} &55.21\text{\scriptsize ±6.19} &62.34\text{\scriptsize ±5.13} &50.91\text{\scriptsize ±2.08}
&54.96\text{\scriptsize ±1.46} &61.68\text{\scriptsize ±0.81} &55.62\text{\scriptsize ±0.32} &68.49\text{\scriptsize ±5.31}
&47.98\text{\scriptsize ±1.11} &55.90\text{\scriptsize ±1.13} &55.18\text{\scriptsize ±2.73} &42.70\text{\scriptsize ±3.26}\\
\method{CPDG} &43.56\text{\scriptsize ±6.41} &65.92\text{\scriptsize ±6.25} &50.32\text{\scriptsize ±5.06} &49.89\text{\scriptsize ±1.34}
&52.86\text{\scriptsize ±0.64} &59.72\text{\scriptsize ±2.53} &53.82\text{\scriptsize ±1.50} &49.71\text{\scriptsize ±2.64}
&47.37\text{\scriptsize ±2.23} &56.40\text{\scriptsize ±1.17} &53.58\text{\scriptsize ±2.10} &40.01\text{\scriptsize ±3.59}\\\midrule
\method{GraphPrompt} &73.78\text{\scriptsize ±5.62} &60.89\text{\scriptsize ±6.37} &64.60\text{\scriptsize ±5.76} &51.28\text{\scriptsize ±2.43}
&55.67\text{\scriptsize ±0.26} &67.46\text{\scriptsize ±0.31} &51.07\text{\scriptsize ±0.75} &\underline{86.78}\text{\scriptsize ±3.14}
&48.46\text{\scriptsize ±0.28} &59.18\text{\scriptsize ±0.49} &50.27\text{\scriptsize ±0.58} &\underline{87.45}\text{\scriptsize ±2.57}\\
\method{ProG} &60.86\text{\scriptsize ±7.43} & 68.60\text{\scriptsize ±5.64}&63.18\text{\scriptsize ±4.79} & 51.46\text{\scriptsize ±2.38}& \underline{92.28}\text{\scriptsize ±0.21} & \underline{93.32}\text{\scriptsize ±0.06}&58.73\text{\scriptsize ±1.58} &86.24\text{\scriptsize ±2.87} & \underline{89.75}\text{\scriptsize ±0.28} & \underline{90.69}\text{\scriptsize ±0.08}&56.42\text{\scriptsize ±1.95} &85.43\text{\scriptsize ±3.16} \\\midrule
\method{TGAT-TIGPrompt} &69.21\text{\scriptsize ±8.88} &67.70\text{\scriptsize ±9.64} &\underline{73.90}\text{\scriptsize ±6.68} &51.38\text{\scriptsize ±2.72}
&59.54\text{\scriptsize ±1.41} &78.45\text{\scriptsize ±1.44} &51.69\text{\scriptsize ±1.24} &69.71\text{\scriptsize ±4.16}
&49.52\text{\scriptsize ±0.85} &65.66\text{\scriptsize ±2.68} &51.58\text{\scriptsize ±4.02} &48.34\text{\scriptsize ±3.28}\\
\method{TGN-TIGPrompt} &44.80\text{\scriptsize ±5.45} &63.75\text{\scriptsize ±5.60} &55.42\text{\scriptsize ±3.60} &50.84\text{\scriptsize ±2.75}
&82.04\text{\scriptsize ±2.03} &83.26\text{\scriptsize ±2.38} &\underline{65.00}\text{\scriptsize ±4.73} &86.25\text{\scriptsize ±2.43}
&81.75\text{\scriptsize ±1.97} &79.51\text{\scriptsize ±2.58} &\underline{64.98}\text{\scriptsize ±4.61} &86.19\text{\scriptsize ±3.06}\\\midrule
TGAT-\model &\textbf{82.09}\text{\scriptsize ±6.43} &\underline{73.50}\text{\scriptsize ±6.47} &\textbf{77.78}\text{\scriptsize ±5.08} &\textbf{52.03}\text{\scriptsize ±2.24}
&69.88\text{\scriptsize ±0.18} &90.76\text{\scriptsize ±0.09} &53.92\text{\scriptsize ±0.97} &72.04\text{\scriptsize ±4.71}
&52.58\text{\scriptsize ±0.23} &75.20\text{\scriptsize ±0.17} &53.29\text{\scriptsize ±0.87} &50.82\text{\scriptsize ±3.67}\\
TGN-\model &\underline{74.47}\text{\scriptsize ±3.44} &\textbf{74.00}\text{\scriptsize ±3.10} &69.06\text{\scriptsize ±3.89} &\underline{51.97}\text{\scriptsize ±2.16}
&\textbf{94.33}\text{\scriptsize ±0.12} &\textbf{96.82}\text{\scriptsize ±0.06} &\textbf{70.17}\text{\scriptsize ±0.75} &\textbf{87.02}\text{\scriptsize ±1.63}
&\textbf{92.22}\text{\scriptsize ±0.19} &\textbf{95.69}\text{\scriptsize ±0.08} &\textbf{69.77}\text{\scriptsize ±0.66} &\textbf{87.63}\text{\scriptsize ±1.97}\\\bottomrule
\hline
\end{tabular}%
}
\parbox{1\linewidth}{\scriptsize Results are reported in percent. The best method is bolded and the runner-up is underlined.}
\end{table*}
% \begin{table*}[tbp]
% \centering
% \addtolength{\tabcolsep}{-1mm}
% \caption{Evaluation on node classification and link prediction in data-abundant setting.}
% \label{table.supervised}
% \resizebox{\textwidth}{!}{%
% \begin{tabular}{@{}l|cccc|cccc|cccc@{}}
% \toprule
% \multirow{2}*{Methods} & \multicolumn{4}{c|}{{Transductive Link Prediction}} & \multicolumn{4}{c|}{{Inductive Link Prediction}} & \multicolumn{4}{c}{{Node Classification}} \\
% \multicolumn{1}{c|}{} & {Wikipedia} & {Reddit} & {MOOC} & {Genre} & {Wikipedia} & {Reddit} & {MOOC} & {Genre} & {Wikipedia} & {Reddit} & {MOOC} & {Genre}\\ \midrule\midrule
% \method{GCN-ROLAND} &58.74\text{\scriptsize ±3.12} &50.57\text{\scriptsize ±2.53} &53.61\text{\scriptsize ±2.45} &53.62\text{\scriptsize ±1.18} & 55.13\text{\scriptsize ±2.31}& 51.42\text{\scriptsize ±2.52}& 51.29\text{\scriptsize ±2.40} &51.45\text{\scriptsize ±1.56} & 63.04\text{\scriptsize ±5.06}&50.68\text{\scriptsize ±4.17} & 49.77\text{\scriptsize ±3.24} &47.82\text{\scriptsize ±1.12}\\
% \method{GAT-ROLAND} &59.83\text{\scriptsize ±2.88} &59.59\text{\scriptsize ±3.08} & 51.61\text{\scriptsize ±2.34} &54.26\text{\scriptsize ±1.88} &55.14\text{\scriptsize ±2.82} & 55.59\text{\scriptsize ±2.98} &48.53\text{\scriptsize ±2.71} &52.74\text{\scriptsize ±1.22} &63.01\text{\scriptsize ±4.97}& 49.95\text{\scriptsize ±4.35}&49.56\text{\scriptsize ±3.16} &48.95\text{\scriptsize ±1.26} \\\midrule
% \method{TGAT} &89.39\text{\scriptsize ±1.35} &94.71\text{\scriptsize ±1.86} &57.85\text{\scriptsize ±2.43} &79.83\text{\scriptsize ±1.62} &87.36\text{\scriptsize ±1.12} &92.51\text{\scriptsize ±1.70} & 57.80\text{\scriptsize ±2.30} &76.58\text{\scriptsize ±2.89} &62.85\text{\scriptsize ±3.55} & 50.04\text{\scriptsize ±3.70}&47.07\text{\scriptsize ±3.43} &52.34\text{\scriptsize ±0.93} \\
% % \method{DyRep} &89.60\text{\scriptsize ±0.12}& 93.87\text{\scriptsize ±0.31}& 66.16\text{\scriptsize ±0.13} &61.47\text{\scriptsize ±3.04} & 87.62\text{\scriptsize ±0.23}&92.57\text{\scriptsize ±0.31} & 66.95\text{\scriptsize ±0.13} &68.25\text{\scriptsize ±3.51}&\underline{77.86\text{\scriptsize ±2.18}} & 52.41\text{\scriptsize ±2.28}& 56.45\text{\scriptsize ±2.16} &51.03\text{\scriptsize ±0.88}\\
% % \method{JODIE} & 88.30\text{\scriptsize ±1.43}& 92.46\text{\scriptsize ±1.59}&67.01\text{\scriptsize ±1.83} &88.52\text{\scriptsize ±2.97} &86.12\text{\scriptsize ±1.33} &92.09\text{\scriptsize ±1.39} &66.55\text{\scriptsize ±1.85} &87.54\text{\scriptsize ±2.67}&76.11\text{\scriptsize ±3.28} &54.54\text{\scriptsize ±3.45} & \textbf{61.13\text{\scriptsize ±2.15}} &52.44\text{\scriptsize ±0.82}\\
% \method{TGN} &96.59\text{\scriptsize ±0.19} & \underline{97.50}\text{\scriptsize ±0.10}& 74.97\text{\scriptsize ±0.19} &92.63\text{\scriptsize ±1.98} & 95.47\text{\scriptsize ±0.24}& \underline{96.62}\text{\scriptsize ±0.18}& 75.20\text{\scriptsize ±0.80} &\underline{88.62}\text{\scriptsize ±1.46} &73.64\text{\scriptsize ±2.61} &59.70\text{\scriptsize ±2.23} &56.64\text{\scriptsize ±2.72} &52.47\text{\scriptsize ±1.12}\\
% \method{Trend} &91.26\text{\scriptsize ±2.36} &94.85\text{\scriptsize ±1.74} &70.26\text{\scriptsize ±2.16} &80.17\text{\scriptsize ±1.92} &87.52\text{\scriptsize ±2.05} &92.75\text{\scriptsize ±1.88} &57.33\text{\scriptsize ±1.14} &77.16\text{\scriptsize ±2.04} &73.71\text{\scriptsize ±1.18} & 53.18\text{\scriptsize ±0.94}&59.89\text{\scriptsize ±2.34} & 57.22\text{\scriptsize ±2.38}\\
% % \method{GraphMixer} & & & & & & & & & & & &\\
% \midrule
% \method{DDGCL} &71.33\text{\scriptsize ±1.54}& 70.85\text{\scriptsize ±1.46}&55.62\text{\scriptsize ±1.53} &78.15\text{\scriptsize ±2.46} & 69.84\text{\scriptsize ±1.46}&69.83\text{\scriptsize ±1.11} &55.57\text{\scriptsize ±1.32} &74.51\text{\scriptsize ±3.74}&48.50\text{\scriptsize ±3.11} &50.66\text{\scriptsize ±3.13} &51.21\text{\scriptsize ±3.73} &51.32\text{\scriptsize ±0.87} \\
% \method{CPDG} &72.06\text{\scriptsize ±1.64} &74.32\text{\scriptsize ±1.25} & 70.51\text{\scriptsize ±2.06} &75.29\text{\scriptsize ±1.82} & 69.75\text{\scriptsize ±1.64}&70.67\text{\scriptsize ±1.53} &70.25\text{\scriptsize ±2.50} &71.36\text{\scriptsize ±1.34} &71.28\text{\scriptsize ±2.23} & \underline{61.99\text{\scriptsize ±2.17}} &\underline{60.50\text{\scriptsize ±2.10}} & 51.69\text{\scriptsize ±1.05}\\\midrule
% \method{GraphPrompt} &\underline{96.73}\text{\scriptsize ±0.26} &96.87\text{\scriptsize ±0.31} &75.42\text{\scriptsize ±0.75} &80.16\text{\scriptsize ±1.27} &\underline{95.51}\text{\scriptsize ±0.28}& 95.14\text{\scriptsize ±0.31}&75.33\text{\scriptsize ±0.58} &77.34\text{\scriptsize ±1.86} &\underline{74.82}\text{\scriptsize ±2.28} & 59.39\text{\scriptsize ±2.49}&57.63\text{\scriptsize ±2.58} & 53.39\text{\scriptsize ±1.22}\\
% \method{ProG} &96.64\text{\scriptsize ±0.65} &97.41\text{\scriptsize ±1.38} &\underline{76.15}\text{\scriptsize ±0.94}& \underline{92.74}\text{\scriptsize ±1.62} &94.50\text{\scriptsize ±0.48}& 96.43\text{\scriptsize ±0.25}&\underline{76.12}\text{\scriptsize ±1.27} &88.43\text{\scriptsize ±1.68} &76.43\text{\scriptsize ±1.86} &62.34\text{\scriptsize ±2.19} & 53.18\text{\scriptsize ±0.94}& 57.21\text{\scriptsize ±1.94} \\
% \midrule
% \method{TGAT-TIGPrompt} & 93.26\text{\scriptsize ±0.88}&96.66\text{\scriptsize ±0.64} &53.61\text{\scriptsize ±1.24} &80.19\text{\scriptsize ±1.12} &87.95\text{\scriptsize ±1.41}&90.40\text{\scriptsize ±1.38} &54.52\text{\scriptsize ±1.24} &76.32\text{\scriptsize ±1.28} &71.91\text{\scriptsize ±3.88} &53.89\text{\scriptsize ±3.68} &50.01\text{\scriptsize ±3.02} &52.83\text{\scriptsize ±1.14} \\
% \method{TGN-TIGPrompt} & 89.03\text{\scriptsize ±1.45}&92.58\text{\scriptsize ±2.60} & 73.60\text{\scriptsize ±1.60} &92.25\text{\scriptsize ±2.04} &88.42\text{\scriptsize ±1.97} & 91.42\text{\scriptsize ±1.38}& 73.72\text{\scriptsize ±1.73} &88.57\text{\scriptsize ±2.44} &53.45\text{\scriptsize ±3.45}&61.44\text{\scriptsize ±3.58}&49.47\text{\scriptsize ±3.60} &53.11\text{\scriptsize ±1.32}\\\midrule
% TGAT-\model &90.21\text{\scriptsize ±0.43} & 95.74\text{\scriptsize ±0.09}& 58.16\text{\scriptsize ±0.83} &81.04\text{\scriptsize ±1.31} &88.37\text{\scriptsize ±0.36} &93.15\text{\scriptsize ±0.58} &58.60\text{\scriptsize ±0.62} &77.63\text{\scriptsize ±1.46}&81.36\text{\scriptsize ±2.23}&\textbf{62.47\text{\scriptsize ±2.17}}&54.70\text{\scriptsize ±2.87} & 54.09\text{\scriptsize ±0.82}\\
% TGN-\model & \textbf{97.32}\text{\scriptsize ±0.44}& \textbf{97.63}\text{\scriptsize ±0.10}&\textbf{77.35}\text{\scriptsize ±1.08} &\textbf{93.61}\text{\scriptsize ±1.58} &\textbf{96.23}\text{\scriptsize ±0.12} & \textbf{96.68}\text{\scriptsize ±0.06}&\textbf{78.16}\text{\scriptsize ±0.75} &\textbf{89.15}\text{\scriptsize ±1.45} &\textbf{77.88}\text{\scriptsize ±2.44} & 59.71\text{\scriptsize ±3.08} &59.40\text{\scriptsize ±2.66} &54.26\text{\scriptsize ±1.06} \\\bottomrule
% \end{tabular}%
% }
% \end{table*}
\stitle{Baselines.}
The performance of \model\ is evaluated against state-of-the-art approaches across four main categories.
%
(1) \emph{Conventional DGNNs}: We first consider a discrete-time framework ROLAND \citep{you2022roland} that adapts static GNNs to DGNNs by treating the node embeddings at each GNN layer as hierarchical node states and updating them recurrently over time. We employ GCN \citep{kipf2016semi} and GAT \citep{velivckovic2017graph} for ROLAND. Additionally, we compare to continuous-time DGNNs, including TGAT \citep{rossi2020temporal}, %DyRep \citep{trivedi2019dyrep},
%JODIE \citep{kumar2019predicting},
TGN \citep{xu2020inductive}, TREND \citep{wen2022trend}, and GraphMixer \citep{cong2023we}.
%\textcolor{blue}{GraphMixer \citep{cong2023we} employ RNN \citep{sherstinsky2020fundamentals}, message passing framework, transformer \citep{kenton2019bert}, or simple MLP} to capture both structural and dynamic information.
For a fair comparison, these conventional DGNNs are pre-trained on the first 80\% of the events as well, and then continually trained on the same training data in each downstream task following our split.
(2) \emph{Dynamic graph pre-training}: DDGCL \citep{tian2021self} and CPDG \citep{bei2023cpdg} follow the ``pre-train, fine-tune'' paradigm. They first pre-train a DGNN, leveraging the intrinsic attributes and dynamic patterns of graphs. Then, they fine-tune the pre-trained model for downstream tasks based on task-specific training data.
(3) \emph{Static graph prompting}: GraphPrompt \citep{liu2023graphprompt} and ProG \citep{sun2023all}
employ a static prompt to adapt pre-trained GNNs to downstream tasks. Specifically, we employ DGNNs as the pre-trained backbones---TGAT for GraphPrompt and TGN for ProG---as these combinations perform competitively with their respective prompting approaches.
%leverage link prediction and a graph-level task during pre-training, respectively, and unify downstream tasks with the pre-training task. Consequently, a static prompt is tuned for downstream adaptation.
(4) \emph{Dynamic graph prompting}: TIGPrompt \citep{chen2024prompt} proposes a prompt generator to generate time-aware prompts.
Note that both TIGPrompt and our method \model\ can be applied to different backbones, such as TGAT and TGN.
%
More details on these baselines are described in Appendix~\ref{app.baselines}, %We adhere to the authors' code and default settings for all open-source baselines and reproduce the non-open-source CPDG and TProG. Additionally, we tune the hyperparameters of all baselines to optimize performance.
with further implementation details for the baselines and \model\ in Appendix~\ref{app.parameters}.
\subsection{Performance comparison with baselines}
We compare the performance of all methods on temporal node classification and link prediction in Table~\ref{table.fewshot} and make the following observations.
% \stitle{Node classification.}
%\textcolor{blue}{data scarce settings} in Table~\ref{table.fewshot}. Further results in data abundant settings are presented in Appendix~\ref{app.exp.abundant}.
%We observe that: %for NC tasks:
First, \model\ achieves outstanding performance across node classification and link prediction tasks.
% Moreover, TGAT-\model\ outperforms all baselines on node classification tasks across all four datasets \textcolor{blue}{in the data-scarce setting} and remains competitive against other baselines on link prediction tasks \textcolor{blue}{in both settings}.
This observation highlights the effectiveness of our proposed dual prompts and condition-nets. %\textcolor{blue}{regardless of the amount of available data}.
To further understand the impact of specific design choices in \model\ and its robustness to various backbones, we defer a detailed investigation to the ablation studies in Sect.~\ref{sec.exp.ablation} and the backbone analysis in Sect.~\ref{sec.exp.backbones}.
%We further conduct a series of ablation studies to evaluate the significance of our specific designs in Sect.~\ref{sec.exp.ablation}, and study the robustness under various backbones
%have a significant impact on the performance, which we further analyze
%in Sect.~\ref{sec.exp.backbones}.
Second, TIGPrompt, the only other dynamic graph prompt learning method, significantly lags behind \model\ when using the same backbone. While it employs time-aware prompts, it lacks fine-grained node-time characterization and is thus unable to capture complex node-time patterns, where nodes and time mutually influence each other.
Third, GraphPrompt and ProG perform competitively because we employ DGNNs as their backbones in pre-training, rather than static GNNs.
%\textcolor{blue}{DGNN (TGAT for GraphPrompt and TGN for ProG) as the backbone, rather than a static GNN}.
However, they still underperform compared to \model, as they only use static prompts.
\textbf{\textsc{Remark.}} It is worth noting that the results of TGAT and TGN in our experiments are lower than those reported in their original papers. A possible reason is that, in their original setting, these methods were trained on data that immediately precede the test set chronologically. In contrast, in our experiments, to maintain consistency with the pre-training setup, we train them in two stages: first with pre-training, and then downstream training on limited task data where we sample only 30 events for each task. Moreover, training data that are temporally closer to the test set are more important than earlier data in pre-training. However, due to limited data in the downstream training stage, the models converge quickly and fail to effectively learn from these crucial recent time intervals, resulting in lower performance than their original setting. In contrast, our prompt-based approach can better handle the temporal gap and adapt well to the test data.
% (3) More advanced "pre-training, finetuning" methods %(DDGCL, CPDG)
% generally outperform DGNN
% %(TGAT, DyRep, JOIDE, TGN)
% , as the former group design more intricate pre-training tasks, thereby obtaining more dynamic knowledge from from label-free dynamic graphs.
% \stitle{Link prediction.}
% For both transductive and inductive LP, we observe similar trends as those observed in the NC results, indicating the generality of \model\ across both NC and LP tasks.
% \begin{figure}[t]
% \centering
% \includegraphics[width=1\linewidth]{figures/visilization-embedding.pdf}
% \caption{Output embedding space of nodes.}
% \label{fig.visualization}
% \end{figure}
\begin{table*}[tb]
\centering
\addtolength{\tabcolsep}{-1mm}
\caption{Ablation study reporting AUC-ROC (\%), with TGAT as the backbone.}
\label{table.ablation}%
\resizebox{1\linewidth}{!}{%
\begin{tabular}{@{}l|cccc|ccc|ccc|ccc@{}}
\toprule
\multirow{2}*{Methods}
& Node & Time & \multirow{2}{*}{$\mathtt{NCN}$} & \multirow{2}{*}{$\mathtt{TCN}$} &\multicolumn{3}{c|}{Node classification} & \multicolumn{3}{c|}{Transductive Link Prediction} & \multicolumn{3}{c}{Inductive Link Prediction}\\
& prompt & prompt & & & Wikipedia & Reddit & MOOC & Wikipedia & Reddit & MOOC & Wikipedia & Reddit & MOOC\\
\midrule\midrule
\method{Variant 1}
& $\times$ & $\times$ & $\times$ &$\times$ &67.00 &53.64 &59.27&55.78& 62.43& 51.49&48.21&57.30 &51.42\\
\method{Variant 2}
& $\checkmark$ & $\times$ & $\times$ &$\times$ &72.59 &61.82 &63.50 &68.12&88.59 &51.24 & 51.89& 74.84&51.37\\
\method{Variant 3}
& $\times$ & $\checkmark$ & $\times$ &$\times$ &73.22 &62.51 &62.59 &66.51 &87.06 &51.26 & 50.28&69.71 &50.50\\
\method{Variant 4}
& $\checkmark$ & $\checkmark$ & $\times$ &$\times$ &72.25 & 63.11&62.87 &68.36 &90.31 &52.17 &52.56 & 75.13&51.33\\
\method{Variant 5}
& $\checkmark$ & $\times$ & $\checkmark$ &$\times$ &81.40 & 73.12&77.15&69.56 &90.10& 52.16 &52.57 & 75.50&53.31\\
\method{Variant 6}
& $\times$ & $\checkmark$ & $\times$ &$\checkmark$ &80.34 & 72.59&76.16&66.62&87.34&51.22 &49.05 &73.16 &52.34\\
% \method{Variant 5}
% & $\times$ & $\times$ & $\checkmark$ &$\times$ &81.66 &72.71 &\textbf{78.80} & 66.22&87.01 &51.61\\
% \method{Variant 6}
% & $\times$ & $\times$ & $\times$ &$\checkmark$ &80.15 &72.61 &77.52 &67.34 &86.96 &51.62\\
% \method{Variant 7}
% & $\times$ & $\times$ & $\checkmark$ &$\checkmark$ &73.76 &64.24 &65.97 & 66.13& 86.98&52.21\\
\method{\model}
& $\checkmark$ & $\checkmark$ & $\checkmark$ & $\checkmark$ & 82.09 &73.50 &77.78 & 69.88 & 90.76 & 53.92 &52.58 &75.20 &53.29\\
\bottomrule
\end{tabular}}
% \parbox{1\linewidth}{\scriptsize NP and TP stand for node prompt and time prompt, respectively. %$\mathtt{NCN}$ and $\mathtt{TCN}$ denote node and time condition-net, respectively.
% }
\end{table*}
\begin{table*}[tbp]
\centering
\small
%\addtolength{\tabcolsep}{-1mm}
\caption{AUC-ROC (\%) evaluation of \model\ with different DGNN backbones.}\vspace{1mm}
\label{table.backbone}%
\resizebox{0.97\linewidth}{!}{%
\begin{tabular}{@{}l|l|ccc|ccc|ccc@{}}
\toprule
{Pre-training} & {Downstream} &\multicolumn{3}{c|}{Node classification} &\multicolumn{3}{c|}{Transductive link prediction} &\multicolumn{3}{c}{Inductive link prediction}\\
Backbone &Adaptation & {Wikipedia} & {Reddit} & {MOOC} & {Wikipedia} & {Reddit} & {MOOC} & {Wikipedia} & {Reddit} & {MOOC}\\
\midrule\midrule
\multirow{2}{*}{\textsc{DyRep}}
& - & 50.61& 49.54& 50.33&58.45 &58.02 & 50.29&56.87 &57.25 &50.34\\
& \method{\model} &\textbf{53.52} &\textbf{50.98} &\textbf{51.62} & \textbf{91.64} & \textbf{93.84} & \textbf{72.04} & \textbf{90.28} & \textbf{93.08} & \textbf{72.27}\\
\midrule
\multirow{2}{*}{JODIE}
& - &51.37 &49.80 &50.53&62.40 & \textbf{59.81} &50.71 & 59.59 &\textbf{61.28} &50.57 \\
& \method{\model} &\textbf{62.84} &\textbf{60.93} &\textbf{67.84} &\textbf{63.56} & 58.89 & \textbf{52.06} & \textbf{62.80} & 58.17 &\textbf{52.33}\\
\midrule
\multirow{2}{*}{TGAT}
& - &67.00 &53.64 &59.27 &55.78& 62.43& 51.49&48.21&57.30 &51.42\\
& \method{\model} &\textbf{82.09} &\textbf{73.50} &\textbf{77.78} & \textbf{69.88} & \textbf{90.76} & \textbf{53.92} & \textbf{52.58} & \textbf{75.20} & \textbf{53.29}\\
\midrule
\multirow{2}{*}{TGN}
& - &50.61 &49.54 &50.33 & 72.48& 67.37&54.60 &74.38 &69.81 & 54.62\\
& \method{\model} &\textbf{74.47} &\textbf{74.00} &\textbf{69.06} &\textbf{94.33} &\textbf{96.82} &\textbf{70.17} &\textbf{92.22} &\textbf{95.69} &\textbf{69.77}\\
\midrule
\multirow{2}{*}{TREND}
& - &69.92&64.85 &66.79 &63.24&\textbf{80.42}&58.70 &50.15&\textbf{65.13}&57.52 \\
& \method{\model} &\textbf{70.15} &\textbf{65.24} & \textbf{67.58}&\textbf{64.35} &79.62&\textbf{59.45}&\textbf{51.26} &64.88&\textbf{59.13} \\
\midrule
\multirow{2}{*}{GraphMixer}
& - & 65.43& 60.21& 63.72& 59.73& 61.88 & 52.42 & 51.34 & \textbf{57.64}&51.16\\
& \method{\model} &\textbf{66.39} & \textbf{61.42}& \textbf{64.18}& \textbf{60.25}& \textbf{62.31}&\textbf{52.94} &\textbf{52.19} &57.43 &\textbf{52.55} \\
\midrule
\end{tabular}}
\parbox{1\linewidth}{\scriptsize \ \ \ ``-'' refers to fine-tuning or continually training the backbone on downstream task data without our prompt design.}
\end{table*}
\subsection{Ablation studies}\label{sec.exp.ablation}
To thoroughly understand the impact of each component within \model, we compare \model\ with its variants that employ different prompts and condition-nets. These variants and their corresponding results, using TGAT as the backbone, are presented in Table~\ref{table.ablation} across three datasets.
%The outcomes indicate that each individual design is instrumental, and we make following observations.
For node classification, both node and time prompts are beneficial, as Variant 2 (with node prompt) and Variant 3 (with time prompt) outperform Variant 1 (without these prompts). Incorporating both prompts in Variant 4 could further boost the performance.
%First, both node and time prompts is essential in bridging task objective gap, as evidenced by the better performance of Variants 2 and 4 compared to Variants 1 and 3, which do not utilize a node prompt.
%Similarly, time prompt is advantageous in narrowing dynamic variations, as Variants 3 and 4 generally outperforms Variants 1 and 2.
Moreover, the node condition-net is advantageous, as it integrates node characteristics into time features, since Variant 5 outperforms Variant 2.
Similarly, the time condition-net is beneficial for capturing temporal subtleties at different times to enhance node feature characterization, as Variant 6 outperforms Variant 3.
%(Note that the node condition-net is built on the node prompt, and the time condition-net is built on the time prompt.)
Lastly, \model\ achieves the best performance, demonstrating the effectiveness of dual prompts and condition-nets.
For link prediction, the results generally exhibit similar patterns to those in node classification. A notable exception is the MOOC dataset, where the differences between the variants are generally smaller. A potential reason is that interactions in MOOC (online courses) are less dynamic and diverse than interactions in Reddit (a major social network) and Wikipedia (a high-traffic site), reducing the effectiveness of prompts designed to bridge the pre-training and downstream gaps. However, our full model \model~still achieves significantly better performance than Variant 1, which does not utilize any prompts. On the other hand, in Reddit and Wikipedia, even though both pre-training and downstream tasks involve link prediction, node prompts remain useful due to the potential distributional differences in the downstream training data across tasks.
To further visualize the effect of our prompt design, we sample 200 suspended nodes and 200 active nodes from \textit{Wikipedia}, \textit{Reddit} and \textit{MOOC}, respectively, and show the output embedding space of these nodes, calculated by Variant 1 (w/o prompts) and \model, in Fig.~\ref{fig.visualization}. We observe that with both dual prompts and dual condition-nets, node embeddings from different classes are clearly separated, demonstrating the advantages of \model.
\subsection{Impact of backbones and hyperparameters}\label{sec.exp.backbones}
First, to analyze the flexibility and robustness of \model, we evaluate its performance on different DGNN backbones, including \textsc{DyRep} \citep{trivedi2019dyrep}, \textsc{JODIE} \citep{kumar2019predicting}, \textsc{TGAT}, \textsc{TGN}, \textsc{TREND}, and \textsc{GraphMixer}.
The results on three datasets are reported in Table~\ref{table.backbone}.
We observe that regardless of the backbone utilized, \model\ surpasses the original backbone without our prompt design in almost all cases, indicating the robustness of our proposed framework. %Additionally, given various performance of different original methods, the choice of which to serve as backbone is essential for final results.
% \begin{figure}[t]
% \centering
% \includegraphics[width=1\linewidth]{figures/para.pdf}
% \caption{Sensitivity of $\alpha$.}
% \label{fig.para}
% \end{figure}
\begin{figure}[t]
\centering
\begin{minipage}[b]{0.7\textwidth}
\centering
\includegraphics[width=0.9\linewidth]{figures/visilization-embedding.pdf}
\vspace{-3mm}
\caption{Visualization of output embedding space of nodes.}
\label{fig.visualization}
\end{minipage}%
\begin{minipage}[b]{0.3\textwidth}
\centering
\includegraphics[width=1\linewidth]{figures/para.pdf}
\vspace{-6mm}
\caption{Sensitivity of $\alpha$.}
\label{fig.para}
\end{minipage}
\end{figure}
%\subsection{Hyperparameter sensitivity}\label{exp.para}
Next, we investigate the sensitivity of hyperparameters in \model. Particularly, we employ two MLPs with a bottleneck structure as the dual condition-nets \citep{wen2023prompt,wu2018reducing}. The perceptrons embed the input feature dimension $d$ to $\tilde{d}$, and then restore it back to $d$, where $\tilde{d}=d/\alpha$. Here, $\alpha$ is a hyperparameter that scales the hidden dimension. We evaluate the impact of $\alpha$, as illustrated in Fig.~\ref{fig.para}.
% Formally, for the time condition-net, given $\vec{f}^\text{time}_t \in \mathbb{R}^d$ for timestamp $t$, the time condition-net generates time-conditioned feature prompt as follows:
% \begin{align}
% \tilde{\vec{p}}^\text{time}_{t,v}=(\vec{f}^\text{time}_{t,v}\vec{W}^{\text{time}}_1+\vec{b}^{\text{time}}_1)\vec{W}^{\text{time}}_2+\vec{b}^{\text{time}}_2,
% \end{align}
% where $\vec{W}^{\text{time}}_1 \in \mathbb{R}^{d, \overline{d}}$, $\vec{W}^{\text{time}}_2 \in \mathbb{R}^{\overline{d}, d}$, $\vec{b}^{\text{time}}_1 \in \mathbb{R}^{\overline{d}}$, $\vec{b}^{\text{time}}_2 \in \mathbb{R}^{d}$ are learnable parameters.
% Similarly, given $\vec{f}^\text{fea}_{t,v} \in \mathbb{R}^d$ for timestamp $t$ node $v$, the feature condition-net generates feature-conditioned time prompt as follows:
% \begin{align}
% \tilde{\vec{p}}^\text{fea}_{t,v}=(\vec{f}^\text{fea}_{t,v}\vec{W}^{\text{fea}}_1+\vec{b}^{\text{fea}}_1)\vec{W}^{\text{fea}}_2+\vec{b}^{\text{fea}}_2,
% \end{align}
% where $\vec{W}^{\text{fea},1} \in \mathbb{R}^{d, \overline{d}}$, $\vec{W}^{\text{fea},2} \in \mathbb{R}^{\overline{d}, d}$, $\vec{b}^{\text{fea},1} \in \mathbb{R}^{\overline{d}}$, $\vec{b}^{\text{fea},2} \in \mathbb{R}^{d}$ are learnable parameters. Here, $\overline{d}$ is the hidden dimension of the condition-net, calculated as $\overline{d}=d/\alpha$, where $\alpha$ is a hyperparameter, zooming in $d$. We assess the impact of $\alpha$, and illustrate its impact in Fig.~\ref{fig.para}.
We observe that for both node classification and link prediction tasks, performance generally declines as $\alpha$ increases, likely due to greater information loss from a ``smaller bottleneck.'' The performance when $\alpha\approx 2$ is generally competitive across different datasets. Therefore, in our experiments, we set $\alpha = 2$.
\section{Conclusions}
In this paper, we explored prompt learning on dynamic graphs, aiming to narrow the gap between pre-training and downstream applications on such graphs. Our proposed approach, \model, employs dual prompts to overcome the divergent objectives and temporal variations across pre-training and downstream tasks. Moreover, we propose dual condition-nets to mutually characterize node and time patterns. Finally, we conducted extensive experiments on four public datasets, demonstrating that \model\ significantly outperforms various state-of-the-art baselines.
\section*{Appendices}
% \subsection{Broader Impact and Limitations}\label{app.limitation}
% \stitle{Broader impact.} Our work explores prompt learning on dynamic graphs. In particular, we observe and recognize the mutual characterization between node and time features, which is important to capturing fine-grained, evolving user-time patterns. To this end, we propose condition-nets to generate conditional prompts, which may inspire the community to pay more attention to the observation that node and time features condition on each other. Such patterns are crucial for various dynamic graph applications such as social network analysis and recommender systems.
% %node-time patterns in dynamic graphs and the characterization of node and time representations.
% Additionally, our proposed \model\ excels in data-scarce settings, which are common in practical applications.
% \stitle{Limitations.} Currently, our pre-training involves only dynamic graphs. However, static graphs are also abundant in the real world. Therefore, a promising future direction is to integrate static graphs into the pre-training phase, and transfer the prior knowledge from both static and dynamic graphs to solve downstream applications.
\section{Algorithm}\label{app.alg}
We outline the key steps for downstream prompt tuning in Algorithm~\ref{alg.prompt}. In lines 6--8, we use dual prompts to modify node and time features. In lines 9--14, we perform dual conditional prompting. Specifically, we generate node conditioned time prompts and time conditioned node prompts (lines 9--11), and then modify node and time's features using these conditional prompt (lines 12--14). In line 15--16, we calculate node embeddings based on the prompted node and time features. Finally, we update the embeddings for the prototypical nodes based on the few-shot labeled data provided in the task (lines 18--19) and optimize dual prompts and dual condition-nets (lines 20--22). Note that updating prototypical nodes is required exclusively for node classification tasks.
\begin{algorithm}[tbp]
\small
\caption{\textsc{Downstream Prompt Tuning for \model}}
\label{alg.prompt}
\begin{algorithmic}[1]
\Require Pre-trained DGNN with parameters $\Theta_0$, including those of $\mathtt{DGE}$ and $\mathtt{TE}$
\Ensure Optimized dual prompts $(\vec{p}^\text{node},\vec{p}^\text{time})$, and optimized parameters $(\kappa,\phi)$ of dual condition-nets
\State \slash* Encoding dynamic graphs and time via pre-trained DGNN *\slash
\State $\vec{p}^\text{node},\vec{p}^\text{time},\kappa,\phi \leftarrow$ initialization
\While{not converged}
\For{each dynamic graph $G=(V, E)$ with feature matrix $\mathbf{X}_t$ at time $t$}
\State $\vec{x}_{t,v} \leftarrow \vec{X}_t[v]$, where $v$ is a node in $G$
\State \slash* Dual prompt modification by Eqs.~\ref{eq.prompt-stru} and \ref{eq.prompt-temp} *\slash
\State $\vec{x}^\text{node}_{t,v} \leftarrow \vec{p}^\text{node}\odot\vec{x}_{t,v}$
\State $\vec{f}^\text{time}_t \leftarrow \vec{p}^\text{time}\odot\vec{f}_t$
\State \slash* Dual conditional prompts generation by Eqs.~\ref{eq.time-condition} and \ref{eq.node-condition} *\slash
\State $\tilde{\vec{p}}^\text{node}_t \leftarrow \mathtt{TCN}(\vec{f}^\text{time}_t;\kappa)$
\State $\tilde{\vec{p}}^\text{time}_{t,v} \leftarrow \mathtt{NCN}(\vec{x}^\text{node}_{t,v};\phi)$
\State \slash* Dual conditional prompts modification by Eqs.~\ref{eq.conditional-prompt-str} and \ref{eq.conditional-prompt-tem} *\slash
\State $\tilde{\vec{x}}^\text{node}_{t,v} \leftarrow \tilde{\vec{p}}^\text{node}_t\odot\vec{x}^\text{node}_{t,v}$
\State $\tilde{\vec{f}}^{\text{time}}_{t,v} \leftarrow \tilde{\vec{p}}^\text{time}_{t,v}\odot\vec{f}^\text{time}_t$
\State \slash* Temporal node embedding calculation by Eq.~\ref{eq.dynamic-graph-encoder} *\slash
\State $\vec{h}_{t,v} \leftarrow \mathtt{DGE} \left( \mathtt{Fuse}(\tilde{\vec{x}}^{\text{node}}_{t,v}, \tilde{\vec{f}}^{\text{time}}_{t,v}), \left\{\mathtt{Fuse}(\tilde{\vec{x}}^{\text{node}}_{t',u}, \tilde{\vec{f}}^{\text{time}}_{t‘,v}) : (u,t') \in \bN_v \right\} \right)$
\State \slash* Update prototypical nodes *\slash
\For{each class $y$}
\State ${\vec{h}}_{t,y} \leftarrow \textsc{Average}(\vec{h}_{t,v}$: instance $v$ belongs to class $y$ at time $t$)
\EndFor
\State \slash* Optimizing dual prompts and the parameters in dual condition-nets *\slash
\State Calculate $\bL_\text{down}(\vec{p}^\text{node},\vec{p}^\text{time},\kappa,\phi)$ by Eq.~\ref{eq.loss}
\State Update $\vec{p}^\text{node},\vec{p}^\text{time},\kappa,\phi$ by backpropagating $\bL_\text{down}(\vec{p}^\text{node},\vec{p}^\text{time},\kappa,\phi)$
\EndFor
\EndWhile
\State \Return $\vec{p}^\text{node},\vec{p}^\text{time},\kappa,\phi$
\end{algorithmic}
\end{algorithm}
\section{Complexity Analysis}\label{complexity}
For a downstream dynamic graph $G=(V,E,T)$, the computational process of \model\ involves two main parts: encoding nodes via a pre-trained TGN and prompt learning. The first part's complexity is determined by the TGN's architecture, similar to other methods using a pre-trained TGN. In a standard TGN, each node aggregates messages from up to $d$ neighbors per layer. Assuming the aggregation involves at most $d$ neighbors, the complexity of calculating node embeddings over $L$ layers per batch time is $O(d^L \cdot |V|)$. The second part, downstream prompt learning, has two stages: dual prompting and dual conditional prompting. For the dual prompting, each node and time feature is adjusted by the node prompt and time prompt, respectively, leading to a complexity of $O(|V|+1)$ per batch time. For the dual condition-nets with $K$ layers, for each timestamp, we first generate the node-based condition prompt with a complexity of $O(K)$, and the time-based condition prompts with a complexity of $O(K \cdot |V|)$. Then the condition prompts are used to modify nodes and time features, resulting in a complexity of $O(|V|+1)$. Therefore, the total complexity for prompt learning per batch time is $O((K+2) \cdot (|V| + 1))$. In conclusion, the overall complexity of \model\ is $O(d^L \cdot |V| + (K+2) \cdot (|V| + 1))$. The first part dominates the overall complexity, as $O(d^L \cdot |V|)$ far exceeds $O((K+2) \cdot (|V| + 1))$. Thus, the additional computational cost introduced by the conditional prompt tuning step is minimal.
\section{Further Descriptions of Datasets} \label{app.dataset}
In this section, we provide a summary of these datasets in Table~\ref{table.dataset} and
a further comprehensive descriptions of these datasets.
\begin{table}[tbp]
\centering
\caption{Summary of datasets.}
\label{table.dataset}
\footnotesize
\resizebox{0.75\linewidth}{!}{%
\begin{tabular}{lcccccc}
\toprule
\multirow{2}*{Dataset} & Nodes & Edges & Node & Dynamic & Feature & Time\\
& num &num &classes &labels &dimension &span\\\midrule
Wikipedia & 9,227 & 157,474 & 2 & 217 & 172 & 30 days\\
Reddit & 11,000 & 672,447 & 2 & 366 & 172 & 30 days\\
MOOC & 7,144 & 411,749 & 2 & 4,066 & 172 & 30 days\\
Genre &1,505 &17,858,395 & 474 & 984 &86 &1,500 days\\
\bottomrule
\end{tabular}}
\end{table}
(1) \textit{Wikipedia}\footnote{\url{http://snap.stanford.edu/jodie/wikipedia.csv}} represents a month of modifications made by contributors on Wikipedia pages \citep{ferschke2012wikipedia}. Following prior studies \citep{rossi2020temporal,xu2020inductive}, we utilize data from the most frequently edited pages and active contributors, obtaining a temporal graph containing 9,227 nodes and 157,474 temporal directed edges. Dynamic labels indicate whether contributors temporarily banned from editing.
(2) \textit{Reddit}\footnote{\url{http://snap.stanford.edu/jodie/reddit.csv}} represents a dynamic network between posts and users on subreddits, where an edge represents a user writes a post to the subreddit, with about 11,000 nodes, about 700,000 temporal edges, and dynamic labels indicating whether user is banned from writing posts.
(3) \textit{MOOC}\footnote{\url{http://snap.stanford.edu/jodie/mooc.csv}} is a student-course dataset, representing the actions taken by student on MOOC platform. The nodes are users and courses, and edges represent the actions by users on the courses. Dynamic labels indicate whether the student drops-out after the action.
(4) \textit{Genre}\footnote{\url{https://object-arbutus.cloud.computecanada.ca/tgb/tgbn-genre.zip}} is a dynamic network linking users to music genres, with edges represents a user listens to a specific genre at a certain time. The dataset includes 1,505 nodes and 17,858,395 temporal edges, and dynamic labels indicate each user's most favored music genre.
\section{Further Descriptions of Baselines} \label{app.baselines}
In this section, we provide additional details about the baselines used in our experiments.
\noindent (1) \textbf{Conventional DGNNs}
\begin{itemize}
\item \textbf{ROLAND} \citep{you2022roland}: ROLAND adapts static GNNs for dynamic graph learning. It offers a new viewpoint for static GNNs, where the node embeddings at different GNN layers are viewed as hierarchical node states, thus capturing temporal evolution.
\item \textbf{TGAT} \citep{rossi2020temporal}: TGAT use a self-attention mechanism and a time encoding technique based on the classical Bochner's theorem from harmonic analysis. By stacking TGAT layers, the network recognizes the node embeddings as functions of time and is able to inductively infer embeddings for both new and observed nodes as the graph evolves.
% \item \textbf{DyRep} \citep{trivedi2019dyrep}: DyRep learns the dynamic representations of nodes based on the evolving graph structures, updating node representations incrementally.
% \item \textbf{JODIE} \citep{kumar2019predicting}: JODIE models each user and item with separate RNNs to update embeddings based on the temporal interactions.
\item \textbf{TGN} \citep{xu2020inductive}: TGN incorporates a memory mechanism that updates node states based on new events, effectively capturing the historical context and dependencies over time. It further propose a novel training strategy that supports efficient parallel processing.
\item \textbf{TREND} \citep{wen2022trend}: TREND incorporates the Hawkes process into graph neural networks, and adopts event dynamics and node dynamics to capture the individual characteristics of each event and the collective characteristics of events on the same node, respectively.
\item \textbf{GraphMixer} \citep{cong2023we}: GraphMixer employs a more simple MLP for feature learning, with a portion of the parameters dedicated to time encoding being fixed. This approach enhances the model's ability to capture temporal dynamics while maintaining a simpler and more flexible architecture.
\end{itemize}
\noindent (2) \textbf{Graph Pre-training Models}
\begin{itemize}
\item \textbf{DDGCL} \citep{tian2021self}: DDGCL designs a self-supervised dynamic graph pre-training method via contrasting two nearby temporal views of the same node identity.
\item \textbf{CPDG} \citep{bei2023cpdg}: CPDG integrates two types of contrastive pre-training strategies to learn comprehensive node representations that encapsulate both long-term and short-term patterns.
\end{itemize}
% \begin{table}[tbp]
% \centering
% \caption{Summary of datasets.}
% \label{table.dataset}
% \footnotesize
% \resizebox{1\linewidth}{!}{%
% \begin{tabular}{@{}lcccccc@{}}
% \toprule
% \multirow{2}*{Dataset} & Nodes & Edges & Node & Dynamic & Feature & Time\\
% & num &num &classes &labels &dimension &span\\\midrule
% Wikipedia & 9,227 & 157,474 & 2 & 217 & 172 & 30 days\\
% Reddit & 11,000 & 672,447 & 2 & 366 & 172 & 30 days\\
% MOOC & 7,144 & 411,749 & 2 & 4,066 & 172 & 30 days\\ \bottomrule
% \end{tabular}}
% \end{table}
\noindent (3) \textbf{Graph Prompt Models}
\begin{itemize}
\item \textbf{GraphPrompt} \citep{liu2023graphprompt}: GraphPrompt employs subgraph similarity as a mechanism to unify different pretext and downstream tasks, including link prediction, node classification, and graph classification. A learnable prompt is subsequently tuned for each downstream task.
\item \textbf{ProG} \citep{sun2023all}: ProG reformulate the node- and edge-level tasks as graph-level tasks, and proposes prompt graphs with specific nodes and structures to guide different task.
\item \textbf{TIGPrompt} \citep{chen2024prompt}: TIGPrompt propose a prompt generator that generates personalized, time-aware prompts for each node, enhancing the adaptability and expressiveness of node embeddings for downstream tasks.
\end{itemize}
\section{Implementation Details} \label{app.parameters}
\stitle{Environment.} The environment in which we run experiments is:
\begin{itemize}
\item Operating system: Windows 11
\item CPU information: 13th Gen Intel(R) Core(TM) i5-13600KF
\item GPU information: GeForce RTX 4070Ti (12 GB)
\end{itemize}
\stitle{Optimizer.}\label{app.general-setting}
For all experiments, we use the Adam optimizer.
\stitle{Details of baselines.}
For all open-source baselines, we utilize the officially provided code and reproduce the non-open-source CPDG and TIGPrompt. Each model is tuned according to the settings recommended in their respective literature to achieve optimal performance.
For both GCN and GAT in Roland, we employ a 2-layer architecture.
For TGAT and TGN, we sample 20 temporal neighbors per node to update their representations.
For Trend, after sampling neighboring nodes, we apply Hawkes processing to these temporal neighbors using different time decay factors based on the time of each event.
For GraphMixer, we employ an MLP to process the input nodes along with their positive and negative examples. The output is then passed through a network composed entirely of linear layers for prediction during training.
For DDGCL, we compares two temporally adjacent views of each node, utilizing a time-dependent similarity evaluation and a GAN-style contrastive loss function.
For CPDG, we employ depth-first-search and breadth-first-search to sample neighbors of each nodes.
For GraphPrompt, we employ 1-hop subgraph for similarity calculation.
For TIGPrompt, we employ projection prompt generator, as it is reported to yield the best performance in their literature.
For all baselines, we set the hidden dimension to 172 for \textit{Wikipedia}, \textit{Reddit}, and \textit{MOOC}, and to 86 for \textit{Genre}. DDGCL, CPDG, and GraphPrompt leverage TGAT as their backbone, while ProG and TIGPrompt use TGN. For TIGPrompt, we evaluate its performance using both TGAT and TGN as the backbone.
% For the baseline GCN \cite{kipf2016semi}, we employ a 3-layer architecture, and set the hidden dimensions to 256.
% For GAT \cite{velivckovic2017graph}, we employ a 2-layer architecture and set the hidden dimension to 64. Additionally, we apply 8 attention heads in the first GAT layer.
% For DGI \cite{velivckovic2017graph}, we utilize a 1-layer GCN as the base model and set the hidden dimensions to 256. Additionally, we employ prelu as the activation function.
% For InfoGraph \cite{sun2019infograph}, a 3-layer GCN is used as the base model, with its hidden dimensions set to 256.
% For GraphCL \cite{you2020graph}, a 1-layer GCN is also employed as its base model, with the hidden dimensions set to 256. Specifically, we select edge dropping as the augmentations, with a default augmentation ratio of 0.2.
% For GPPT \cite{sun2022gppt}, we utilize a 2-layer GraphSAGE as its base model, setting the hidden dimensions to 256. For base GraphSAGE, we also employ a mean aggregator.
% For GraphPrompt \cite{liu2023graphprompt}, a 3-layer GCN is used as the base model for all datasets, with the hidden dimensions set to 256.
% For GCOPE \cite{zhao2024all},We employ a 2-layer GCN as the base model and set the unify dimensions and hidden dimensions to 100.
\stitle{Details of \model.}
For our proposed \model, We conducted experiments using TGN and TGAT as backbones. We employ a dual-layer perceptrons with bottleneck structure as the condition-net, and set the hidden dimension of the condition net as 86 for \textit{Wikipedia}, \textit{Reddit} and \textit{MOOC}, while 43 for \textit{Genre}. We set the hidden dimension to 172 for \textit{Wikipedia}, \textit{Reddit}, and \textit{MOOC}, and to 86 for \textit{Genre}.
% \subsection{Further Experiments and Analysis}\label{app.exp}
% \subsection{Performance evaluation in data abundant setting} \label{app.exp.abundant}
% To further evaluate the performance of \model, we first sample 1\% of the data as the training set and another 1\% as the validation set for data abundant setting, with the remaining data used for testing. We evaluate the performance of \model\ on node classification and link prediction tasks on \textit{Wikipedia}, \textit{Reddit}, \textit{MOOC} and \textit{Genre}, and illustrate the results in Table~\ref{table.supervised}. We observe that \model\ consistently outperforms all the other baselines, further showing the effectiveness of our design.
% \begin{table*}[tbp]
% \centering
% \caption{Accuracy evaluation on node classification and link prediction in data abundant setting.}
% \label{table.supervised}
% \resizebox{\textwidth}{!}{%
% \begin{tabular}{@{}l|ccc|ccc|ccc@{}}
% \toprule
% \multirow{2}*{Methods} & \multicolumn{3}{c|}{{Transductive Link Prediction}} & \multicolumn{3}{c|}{{Inductive Link Prediction}} & \multicolumn{3}{c}{{Node Classification}} \\
% \multicolumn{1}{c|}{} & {Wikipedia} & {Reddit} & {MOOC} & {Wikipedia} & {Reddit} & {MOOC} & {Wikipedia} & {Reddit} & {MOOC} \\ \midrule\midrule
% \method{GCN-ROLAND} &58.74 &50.57 &53.61 & 55.13& 51.42& 51.29& 63.04&50.68 & 49.77\\
% \method{GAT-ROLAND} &59.83 &59.59 & 51.61&55.14 & 55.59&48.53 &63.01& 49.95&49.56\\\midrule
% \method{TGAT} &89.39 &94.71 &57.85 &87.36 &92.51 & 57.80&62.85 & 50.04&47.07\\
% \method{DyRep} &89.60& 93.87& 66.16& 87.62&92.57 & 66.95&\underline{77.86} & 52.41& 56.45\\
% \method{JODIE} & 88.30& 92.46&\underline{67.01} &86.12 &92.09 &\underline{66.55} &76.11 &54.54 & \textbf{61.13}\\
% \method{TGN} &96.59 & \underline{97.50}& 74.97& \underline{95.47}& \underline{96.62}& 75.20&73.64 &59.70 &56.64\\\midrule
% \method{DDGCL} &71.33& 70.85&55.62 & 69.84&69.83 &55.57 &48.50 &50.66 &51.21 \\
% \method{CPDG} &72.06 &74.32 & 70.51& 69.75&70.67 &70.25 &71.28 & \underline{61.99} &\underline{60.50} \\\midrule
% \method{GraphPrompt} &\underline{96.73} &96.87 &75.42 &95.51& 95.14&75.33 &74.82 & 59.39&57.63\\\midrule
% \method{TGAT-TIGPrompt} & 93.26&96.66 &53.61 &87.95&90.40 &54.52 &71.91 &53.89 &50.01\\
% \method{TGN-TIGPrompt} & 89.03&92.58 & 73.60 &88.42 & 91.42& 73.72&53.45&61.44&49.47\\\midrule
% TGAT-\model &90.21 & 95.74& 56.25&86.32 &88.02 &55.58&80.36&\textbf{62.47}&54.70 \\
% TGN-\model & \textbf{97.32}& \textbf{97.63}&\textbf{77.35} &\textbf{96.23} & \textbf{96.68}&\textbf{78.16} &\textbf{77.88} & 59.71 &59.40\\\bottomrule
% \end{tabular}%
% }
% \end{table*}
% \stitle{Ablation analysis on inductive link prediction tasks} \label{app.exp.ablation}
% We further conduct ablation study on inductive link prediction, and illustrate the results in Table~\ref{app.exp.ablation}, generally showing the same patterns as we observed in Table~\ref{table.ablation}.
% \begin{table}[tb]
% \centering
% \caption{Ablation study on inductive link prediction tasks.}
% \label{table.app.ablation}
% \resizebox{0.8\linewidth}{!}{
% \begin{tabular}{@{}l|cccc|ccc@{}}
% \toprule
% \multirow{2}{*}{Methods}
% & Feature & Time & Feature & Time &\multicolumn{3}{c}{Indictive link prediction} \\
% &prompt &prompt &condition-net & condition-net & Wikipedia & Reddit & MOOC \\
% \midrule\midrule
% \method{Variant 1}
% & $\times$ & $\times$ & $\times$ &$\times$&52.27&72.90 &51.42\\
% \method{Variant 2}
% & $\checkmark$ & $\times$ & $\times$ &$\times$ & 51.89& 74.84&51.37\\
% \method{Variant 3}
% & $\times$ & $\checkmark$ & $\times$ &$\times$ & 50.28&69.71 &50.50\\
% \method{Variant 4}
% & $\checkmark$ & $\checkmark$ & $\times$ &$\times$ &52.56 & 75.13&51.33\\
% % \method{Variant 5}
% % & $\times$ & $\times$ & $\checkmark$ &$\times$ &48.79 &66.18 &51.59\\
% % \method{Variant 6}
% % & $\times$ & $\times$ & $\times$ &$\checkmark$ &51.95 &66.90 &51.38\\
% % \method{Variant 7}
% % & $\times$ & $\times$ & $\checkmark$ &$\checkmark$ &48.93 &75.15 &51.52 \\
% \method{Variant 5}
% & $\checkmark$ & $\times$ & $\checkmark$ &$\times$ &52.57 & \textbf{75.50}&53.31\\
% \method{Variant 6}
% & $\times$ & $\checkmark$ & $\times$ &$\checkmark$ &49.05 &73.16 &52.34\\
% \method{\model}
% & $\checkmark$ & $\checkmark$ & $\checkmark$ & $\checkmark$&\textbf{52.58} &75.20 &\textbf{53.29}\\
% \bottomrule
% \end{tabular}}
% \end{table}
% \begin{figure}[t]
% \centering
% \includegraphics[width=1\linewidth]{figures/para.png}
% \caption{Impact of hyperparameter $\alpha$ in dual condition-nets.}
% \label{fig.para}
% \end{figure}
% \subsection{Data Ethics Statement}
% To assess the effectiveness of our approach, we performed experiments exclusively using publicly available datasets, including Wikipedia, Reddit, MOOC, and Genre, following their respective terms and conditions. We also confirm that no personally identifiable information was utilized, and no human or animal subjects were involved in this research.
% \begin{table}[tbp]
% \centering
% \caption{\yu{AUC-ROC (\%) evaluation of \model\ on large-scale dataset DGraph.}}
% \label{table.large-dataset}
% \footnotesize
% \resizebox{0.9\linewidth}{!}{%
% \begin{tabular}{@{}l|ccc@{}}
% \toprule
% Method & Node Classification & Transductive Link Prediction & Inductive Link Prediction \\\midrule
% \method{GraphPrompt} &60.18 & & \\
% \method{TIGPrompt} &64.42 & & \\
% \model &\textbf{73.39} & & \\
% \bottomrule
% \end{tabular}}
% \end{table}
\section{Additional Experiments}
\subsection{Performance on large-scale dataset}
We evaluate \model\ under the same setting introduced in Sect.~\ref{sec:expt:setup} on a large-scale dataset DGraph \citep{huang2022dgraph}, which consists of 3,700,550 nodes, 4,300,999 edges, and 1,225,601 labeled nodes. The dataset is designed for the anomaly detection task, a form of binary node classification. The results are shown in Table~\ref{table.large-dataset}. We observe that \model\ outperforms competitive baselines, showing the effectiveness of \model\ on large-scale datasets.
\begin{table}[tbp]
\centering
\caption{AUC-ROC (\%) evaluation for anomaly detection on a large-scale dataset DGraph.}
\label{table.large-dataset}
\footnotesize
%\resizebox{0.35\linewidth}{!}{%
\begin{tabular}{l|c}
\toprule
Method & DGraph \\\midrule
\method{GraphPrompt} &60.18 $\pm$ 13.06\\
\method{TIGPrompt} &64.42 $\pm$ 10.37\\
\model &\textbf{73.39} $\pm$ \ \ 5.85\\
\bottomrule
\end{tabular}%}
\end{table}
\subsection{Efficiency analysis}
We compare the prompt tuning and testing runtimes between \model\ and competitive baselines for the node classification task. As shown in Table~\ref{table.time-nc}, \model\ takes only marginally longer than GraphPrompt, while requiring less time than TIGPrompt. The slight overhead over GraphPrompt is acceptable given the
substantial improvement in performance. Note that the testing runtimes are measured on the entire test set, which can be much larger than the samples used for prompt tuning, resulting in generally longer runtimes compared to prompt tuning.
% \begin{table}[tbp]
% \centering
% \caption{\yu{Training and testing time (seconds) on node classification.}}
% \label{table.time-lp}
% \footnotesize
% \resizebox{0.8\linewidth}{!}{%
% \begin{tabular}{@{}l|cc|cc|cc@{}}
% \toprule
% \multirow{2}*{Method} &\multicolumn{2}{c|}{Wikipedia} &\multicolumn{2}{c|}{Reddit} &\multicolumn{2}{c}{DGraph} \\
% &Training &Testing &Training &Testing &Training &Testing\\\midrule
% \method{GraphPrompt} & & & &&0.436 &0.324\\
% \method{TIGPrompt} & & & &&0.847 &1.063\\
% \model & & & & &0.574 &0.633\\
% \bottomrule
% \end{tabular}}
% \end{table}
\begin{table}[tbp]
\centering
\caption{Prompt-tuning and testing runtimes (seconds) on node classification.}
\label{table.time-nc}
\footnotesize
%\resizebox{0.8\linewidth}{!}{%
\begin{tabular}{l|cc|cc|cc}
\toprule
\multirow{2}*{Method} &\multicolumn{2}{c|}{Wikipedia} &\multicolumn{2}{c|}{Reddit} &\multicolumn{2}{c}{DGraph} \\
&Training &Testing &Training &Testing &Training &Testing\\
\midrule
\method{GraphPrompt} &0.147 &0.230 &0.128 &0.251 &0.436 &0.324\\
\method{TIGPrompt} &0.312 &0.397 &0.356 &0.411 &0.847 &1.063\\
\model &0.273 &0.233 &0.134 &0.274 &0.574 &0.633\\
\bottomrule
\end{tabular}%}
\end{table}
% \begin{table}[tbp]
% \centering
% \caption{\yu{Prompt-tuning and testing times (seconds) on transductive link prediction.}}
% \label{table.time-lp}
% \footnotesize
% %\resizebox{0.6\linewidth}{!}{%
% \begin{tabular}{l|cc|cc}
% \toprule
% \multirow{2}*{Method} &\multicolumn{2}{c|}{Wikipedia} &\multicolumn{2}{c}{Reddit} \\
% &Training &Testing &Training &Testing \\\midrule
% \method{GraphPrompt} &0.841 &7.595 &1.127 &28.812 \\
% \method{TIGPrompt} &1.426 &8.289 &1.541 &38.672\\
% \model &0.873 &7.761 &1.142 &35.859 \\
% \bottomrule
% \end{tabular}%}
% \end{table}
\subsection{Performance using the splits of TIGPrompt}
We adopt the splits of TIGPrompt and conduct further experiments, using 50\% of the data for pre-training and 20\% for prompt tuning, with the remaining 30\% equally divided for validation and testing. We report the results in Table~\ref{table.tigprompt-setting}, where \model\ still consistently outperforms TIGPrompt.
% \begin{table}[tbp]
% \centering
% \caption{\yu{AUC-ROC (\%) evaluation using the splits of TIGPrompt \citep{chen2024prompt}.}}
% \label{table.tigprompt-setting}
% \footnotesize
% \resizebox{0.9\linewidth}{!}{%
% \begin{tabular}{@{}l|ccc|ccc@{}}
% \toprule
% \multirow{2}*{Method} &\multicolumn{3}{c|}{Wikipedia} &\multicolumn{3}{c}{MOOC}\\
% & NC & Transductive LP & Inductive LP & NC & Transductive LP & Inductive LP\\\midrule
% \method{TIGPrompt} &78.85\text{\scriptsize±1.35} &93.95\text{\scriptsize ±0.47} &91.35\text{\scriptsize ±0.38} &63.40 \text{\scriptsize±2.31}&78.98 \text{\scriptsize±0.52}&80.26\text{\scriptsize±0.76}\\
% \model &\textbf{81.31} \text{\scriptsize ±1.13}&\textbf{97.78}\text{\scriptsize±0.36} &\textbf{96.73}\text{\scriptsize±0.42} &\textbf{64.58}\text{\scriptsize±1.95} &\textbf{87.15}\text{\scriptsize±0.42} &\textbf{86.14}\text{\scriptsize±0.39}\\
% \bottomrule
% \end{tabular}}
% \parbox{0.9\linewidth}{\footnotesize NC is short for node classification, and lp is short for link prediction.}
% \end{table}
\begin{table}[t]
\centering
\caption{AUC-ROC (\%) evaluation using the splits of TIGPrompt \citep{chen2024prompt}.}
\label{table.tigprompt-setting}
\footnotesize
%\resizebox{0.9\linewidth}{!}{%
\begin{tabular}{l|cc|cc|cc}
\toprule
\multirow{2}*{Method} &\multicolumn{2}{c|}{Node classification} &\multicolumn{2}{c|}{Transductive link prediction} &\multicolumn{2}{c}{Inductive link prediction}\\
& Wikipedia & MOOC& Wikipedia & MOOC & Wikipedia & MOOC\\\midrule
\method{TIGPrompt} &78.85\text{\scriptsize±1.35} &63.40\text{\scriptsize±2.31} &93.95\text{\scriptsize ±0.47} & 78.98\text{\scriptsize±0.52} &91.35\text{\scriptsize ±0.38} &80.26\text{\scriptsize±0.76}\\
\model &\textbf{81.31}\text{\scriptsize ±1.13} &\textbf{64.58}\text{\scriptsize±1.95} &\textbf{97.78}\text{\scriptsize±0.36} &\textbf{87.15}\text{\scriptsize±0.42} &\textbf{96.73}\text{\scriptsize±0.42} &\textbf{86.14}\text{\scriptsize±0.39}\\
\bottomrule
\end{tabular}%}
\end{table}
\begin{table}[tbp]
\centering
\caption{AUC-ROC (\%) evaluation for five-way node classification on ML-Rating.}
\label{table.mlrating}
\footnotesize
%\resizebox{0.35\linewidth}{!}{%
\begin{tabular}{l|c}
\toprule
Method & ML-Rating \\\midrule
\method{GraphPrompt} &52.37 $\pm$ 1.29\\
\method{TIGPrompt} &53.26 $\pm$ 1.33\\
\model &\textbf{54.82} $\pm$ 1.27\\
\bottomrule
\end{tabular}%}
\end{table}
\subsection{Performance on five-way classification}
While most of our datasets involve binary classification,
we also conduct five-way node classification on the ML-Rating dataset \citep{harper2015movielens} with 9,995 nodes, 1,000,210 edges, and 5 classes. We adopt the same setting introduced in Sect.~\ref{sec:expt:setup} and compare \model\ with two competitive baselines in Table~\ref{table.mlrating}. \model\ consistently outperforms these baselines, demonstrating its effectiveness.
\section{Further Comparison with Related Work}
We summarize the comparison of \model\ with representative graph prompt learning methods, including GraphPrompt \citep{liu2023graphprompt} and ProG \citep{sun2023all} for static graphs, and the contemporary work TIGPrompt \citep{chen2024prompt} for dynamic graphs (using its best-performing variant, Projection TProG, for illustration), as shown in Table~\ref{table.comparison}.
More specifically, first, a node prompt modifies the node features to reformulate the task input and bridge the task gap (Eq.~\ref{eq.prompt-stru}), while a time prompt adjusts the time features to capture the temporal evolution of the dynamic graph (Eq.~\ref{eq.prompt-temp}). The dual prompts are motivated by Challenge 1 in Sect.~\ref{sec.intro}. Previous static graph prompt learning methods (GraphPrompt and ProG) only utilize a node prompt, neglecting the temporal gap between pre-training and downstream tasks. Furthermore, TIGPrompt, falls short of explicitly addressing each gap individually.
The dual prompts also demonstrate empirical benefits. As shown in the ablation study in Table~\ref{table.ablation}, Variant 2 (using only node prompt) and Variant 3 (using only time prompt) consistently outperform Variant 1. Additionally, Variant 4 (incorporating both node and time prompts) generally outperforms Variants 2 and 3, demonstrating the effectiveness of node and time prompts in bridging the task gap and temporal gap between pre-training and downstream tasks.
Second, time-aware node prompts adjust node prompts, and ultimately node features, to reflect temporal influence on the node prompts (Eq.~\ref{eq.conditional-prompt-str}) instead of using a fixed node prompt across time. On the other hand, node-aware time prompts adjust the time prompt for each node by incorporating node-specific characteristics (Eq.~\ref{eq.conditional-prompt-tem}) instead of using a fixed time prompt across nodes. Consequently, node-time patterns and their dynamic interplay are captured, addressing the Challenge 2 in Section 1. Static graph prompt learning methods lack such designs, failing to capture the mutual characterization between node and time patterns. Moreover, TIGPrompt considers only the temporal impact on nodes by generating time-aware node prompts, but neglects that time prompts for each node can also be influenced by node features, hindering its ability to model the scenario where different nodes may exhibit divergent behaviors even at the same time point, as illustrated in Fig.~\ref{fig.intro-motivation}(a).
As shown in the ablation study in Table 2, Variant 5 (with time-aware node prompts) outperforms Variant 2 (without time-aware node prompts), Variant 6 (with node-aware time prompts) outperforms Variant 3 (without node-aware time prompts), and DyGPrompt (with both) outperforms Variant 4 (without either), further demonstrating the effectiveness of our designs.
Third, we propose dual condition-nets to effectively generate time-aware node prompts (Eq.~\ref{eq.time-condition}) and node-aware time prompts (Eq.~\ref{eq.node-condition}) while avoid overfitting in a parameter-efficient way. These condition-nets generate prompts conditioned on the input features rather than directly parameterizing the prompts, significantly reducing the number of learnable parameters in the downstream prompt-tuning phase.
In our implementation, we use a 2-layer MLP as the condition-net. Specifically, our dual condition-nets contains merely 3,104 learnable parameters for Wikipedia, Reddit and MOOC, 1,554 for Genre and 8,100
for DGraph, to generate all the time-aware node prompts and node-aware time prompts.
In contrast, to generate time-aware node prompts, TIGPrompt requires learning a specific prompt for each node, leading to a significantly larger number of learnable parameters: 1,609,274 for Wikipedia, 1,911,478 for Reddit, 1,250,998 for MOOC, 130,826 for Genre, and 66,610,156 for DGraph. Moreover, as shown in Table~\ref{table.time-nc}, \model\ requires less training and testing time compared to TIGPrompt, further demonstrating the efficiency of using condition-net. Thus, the condition-nets in \model\ is another significant difference from previous work, addressing Challenge 2 in a parameter-efficient manner.
\begin{table}[tbp]
\centering
\caption{Comparison with representative prompt learning methods.}
\label{table.comparison}
\footnotesize
\resizebox{0.8\linewidth}{!}{%
\begin{tabular}{@{}l|ccccc@{}}
\toprule
& \makecell{Explicit\\node prompt} & \makecell{Explicit\\time prompt} & \makecell{Time-aware\\node prompts} & \makecell{Node-aware\\time prompts} &Condition-net\\\midrule
\method{GraphPrompt} &$\checkmark$ &$\times$ &$\times$ &$\times$ &$\times$ \\
\method{ProG} &$\checkmark$ &$\times$ &$\times$ &$\times$ &$\times$ \\
\method{TIGPrompt} &$\times$ &$\times$ &$\checkmark$ &$\times$ &$\times$\\
\model &$\checkmark$ &$\checkmark$ &$\checkmark$ &$\checkmark$ &$\checkmark$\\
\bottomrule
\end{tabular}}
\end{table}
\begin{thebibliography}{66}
\providecommand{\natexlab}[1]{#1}
\providecommand{\url}[1]{\texttt{#1}}
\expandafter\ifx\csname urlstyle\endcsname\relax
\providecommand{\doi}[1]{doi: #1}\else
\providecommand{\doi}{doi: \begingroup \urlstyle{rm}\Url}\fi
\bibitem[Barros et~al.(2021)Barros, Mendon{\c{c}}a, Vieira, and Ziviani]{barros2021survey}
Claudio~DT Barros, Matheus~RF Mendon{\c{c}}a, Alex~B Vieira, and Artur Ziviani.
\newblock A survey on embedding dynamic graphs.
\newblock \emph{ACM Computing Surveys}, 55\penalty0 (1), 2021.
\bibitem[Bei et~al.(2024)Bei, Xu, Zhou, Chi, Zhang, Li, and Bu]{bei2023cpdg}
Yuanchen Bei, Hao Xu, Sheng Zhou, Huixuan Chi, Mengdi Zhang, Zhao Li, and Jiajun Bu.
\newblock Cpdg: A contrastive pre-training method for dynamic graph neural networks.
\newblock \emph{ICDE}, pp.\ 1199--1212, 2024.
\bibitem[Brown et~al.(2020)Brown, Mann, Ryder, Subbiah, Kaplan, Dhariwal, Neelakantan, Shyam, Sastry, Askell, et~al.]{brown2020language}
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared~D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et~al.
\newblock Language models are few-shot learners.
\newblock In \emph{NeurIPS}, pp.\ 1877--1901, 2020.
\bibitem[Chen et~al.(2022)Chen, Zhang, Jiang, Wang, and Dai]{chen2022pre}
Ke-Jia Chen, Jiajun Zhang, Linpu Jiang, Yunyun Wang, and Yuxuan Dai.
\newblock Pre-training on dynamic graph neural networks.
\newblock \emph{Neurocomputing}, 500\penalty0 (C):\penalty0 679--687, 2022.
\bibitem[Chen et~al.(2024)Chen, Zhang, Xiong, Wu, Zhang, Sun, Zhang, Zhao, and Kang]{chen2024prompt}
Xi~Chen, Siwei Zhang, Yun Xiong, Xixi Wu, Jiawei Zhang, Xiangguo Sun, Yao Zhang, Yinglong Zhao, and Yulin Kang.
\newblock Prompt learning on temporal interaction graphs.
\newblock \emph{arXiv preprint arXiv:2402.06326}, 2024.
\bibitem[Cong et~al.(2022)Cong, Zhang, Kang, Yuan, Wu, Zhou, Tong, and Mahdavi]{cong2022we}
Weilin Cong, Si~Zhang, Jian Kang, Baichuan Yuan, Hao Wu, Xin Zhou, Hanghang Tong, and Mehrdad Mahdavi.
\newblock Do we really need complicated model architectures for temporal networks?
\newblock In \emph{ICLR}, 2022.
\bibitem[Cong et~al.(2023)Cong, Zhang, Kang, Yuan, Wu, Zhou, Tong, and Mahdavi]{cong2023we}
Weilin Cong, Si~Zhang, Jian Kang, Baichuan Yuan, Hao Wu, Xin Zhou, Hanghang Tong, and Mehrdad Mahdavi.
\newblock Do we really need complicated model architectures for temporal networks?
\newblock In \emph{ICLR}, 2023.
\bibitem[Duan et~al.(2024)Duan, Jiang, Zhang, Wang, Peng, Ziqi, Mao, Wu, and Wang]{duan2024cat}
Yifan Duan, Xinke Jiang, Guibin Zhang, Shilong Wang, Xiaojiang Peng, Wang Ziqi, Junyuan Mao, Hao Wu, and Kun Wang.
\newblock {CaT-GNN}: Enhancing credit card fraud detection via causal temporal graph neural networks.
\newblock \emph{arXiv preprint arXiv:2402.14708}, 2024.
\bibitem[Fan et~al.(2019)Fan, Ma, Li, He, Zhao, Tang, and Yin]{fan2019graph}
Wenqi Fan, Yao Ma, Qing Li, Yuan He, Eric Zhao, Jiliang Tang, and Dawei Yin.
\newblock Graph neural networks for social recommendation.
\newblock In \emph{WWW}, pp.\ 417--426, 2019.
\bibitem[Fang et~al.(2024)Fang, Zhang, Yang, Wang, and Chen]{fang2022universal}
Taoran Fang, Yunchao Zhang, Yang Yang, Chunping Wang, and Lei Chen.
\newblock Universal prompt tuning for graph neural networks.
\newblock In \emph{NeurIPS}, pp.\ 52464--52489, 2024.
\bibitem[Ferschke et~al.(2012)Ferschke, Zesch, and Gurevych]{ferschke2012wikipedia}
Oliver Ferschke, Torsten Zesch, and Iryna Gurevych.
\newblock Wikipedia revision toolkit: Efficiently accessing {W}ikipedia’s edit history.
\newblock \emph{Modern Information Technologies and Innovation Methodologies of Education in Professional Training Methodology Theory Experience Problems}, 2012.
\bibitem[Ha et~al.(2022)Ha, Dai, and Le]{ha2022hypernetworks}
David Ha, Andrew~M Dai, and Quoc~V Le.
\newblock Hypernetworks.
\newblock In \emph{ICLR}, 2022.
\bibitem[Harper \& Konstan(2015)Harper and Konstan]{harper2015movielens}
F~Maxwell Harper and Joseph~A Konstan.
\newblock The movielens datasets: History and context.
\newblock \emph{TIIS}, 5\penalty0 (4):\penalty0 1--19, 2015.
\bibitem[Hu et~al.(2020{\natexlab{a}})Hu, Liu, Gomes, Zitnik, Liang, Pande, and Leskovec]{hu2019strategies}
Weihua Hu, Bowen Liu, Joseph Gomes, Marinka Zitnik, Percy Liang, Vijay Pande, and Jure Leskovec.
\newblock Strategies for pre-training graph neural networks.
\newblock In \emph{ICLR}, 2020{\natexlab{a}}.
\bibitem[Hu et~al.(2020{\natexlab{b}})Hu, Dong, Wang, Chang, and Sun]{hu2020gpt}
Ziniu Hu, Yuxiao Dong, Kuansan Wang, Kai-Wei Chang, and Yizhou Sun.
\newblock {GPT-GNN}: Generative pre-training of graph neural networks.
\newblock In \emph{SIGKDD}, pp.\ 1857--1867, 2020{\natexlab{b}}.
\bibitem[Huang et~al.(2022)Huang, Yang, Wang, Wang, Zhang, Xu, Chen, and Vazirgiannis]{huang2022dgraph}
Xuanwen Huang, Yang Yang, Yang Wang, Chunping Wang, Zhisheng Zhang, Jiarong Xu, Lei Chen, and Michalis Vazirgiannis.
\newblock {DGraph}: A large-scale financial dataset for graph anomaly detection.
\newblock In \emph{NeurIPS}, pp.\ 22765--22777, 2022.
\bibitem[Iba et~al.(2010)Iba, Nemoto, Peters, and Gloor]{iba2010analyzing}
Takashi Iba, Keiichi Nemoto, Bernd Peters, and Peter~A Gloor.
\newblock Analyzing the creative editing behavior of wikipedia editors: Through dynamic social network analysis.
\newblock \emph{Procedia - Social and Behavioral Sciences}, 2\penalty0 (4):\penalty0 6441--6456, 2010.
\bibitem[Jia et~al.(2021)Jia, Yang, Xia, Chen, Parekh, Pham, Le, Sung, Li, and Duerig]{jia2021scaling}
Chao Jia, Yinfei Yang, Ye~Xia, Yi-Ting Chen, Zarana Parekh, Hieu Pham, Quoc Le, Yun-Hsuan Sung, Zhen Li, and Tom Duerig.
\newblock Scaling up visual and vision-language representation learning with noisy text supervision.
\newblock In \emph{ICLR}, 2021.
\bibitem[Jiang et~al.(2023)Jiang, Qin, Xu, and Ao]{jiang2023incomplete}
Xinke Jiang, Zidi Qin, Jiarong Xu, and Xiang Ao.
\newblock Incomplete graph learning via attribute-structure decoupled variational auto-encoder.
\newblock In \emph{WSDM}, pp.\ 304--312. 2023.
\bibitem[Kipf \& Welling(2017)Kipf and Welling]{kipf2016semi}
Thomas~N Kipf and Max Welling.
\newblock Semi-supervised classification with graph convolutional networks.
\newblock In \emph{ICLR}, 2017.
\bibitem[Kumar et~al.(2015)Kumar, Spezzano, and Subrahmanian]{kumar2015vews}
Srijan Kumar, Francesca Spezzano, and VS~Subrahmanian.
\newblock {VEWS}: A wikipedia vandal early warning system.
\newblock In \emph{SIGKDD}, pp.\ 607--616, 2015.
\bibitem[Kumar et~al.(2018)Kumar, Hamilton, Leskovec, and Jurafsky]{kumar2018community}
Srijan Kumar, William~L Hamilton, Jure Leskovec, and Dan Jurafsky.
\newblock Community interaction and conflict on the web.
\newblock In \emph{WWW}, pp.\ 933--943, 2018.
\bibitem[Kumar et~al.(2019)Kumar, Zhang, and Leskovec]{kumar2019predicting}
Srijan Kumar, Xikun Zhang, and Jure Leskovec.
\newblock Predicting dynamic embedding trajectory in temporal interaction networks.
\newblock In \emph{SIGKDD}, pp.\ 1269--1278, 2019.
\bibitem[Le-Khac et~al.(2020)Le-Khac, Healy, and Smeaton]{le2020contrastive}
Phuc~H Le-Khac, Graham Healy, and Alan~F Smeaton.
\newblock Contrastive representation learning: A framework and review.
\newblock \emph{IEEE Access}, 8:\penalty0 193907--193934, 2020.
\bibitem[Lee et~al.(2019)Lee, Im, Jang, Cho, and Chung]{lee2019melu}
Hoyeop Lee, Jinbae Im, Seongwon Jang, Hyunsouk Cho, and Sehee Chung.
\newblock {MeLU}: Meta-learned user preference estimator for cold-start recommendation.
\newblock In \emph{SIGKDD}, pp.\ 1073--1082, 2019.
\bibitem[Li et~al.(2022)Li, Jiang, Zhong, Trajcevski, Wu, and Zhou]{li2022mining}
Rongfan Li, Xinke Jiang, Ting Zhong, Goce Trajcevski, Jin Wu, and Fan Zhou.
\newblock Mining spatio-temporal relations via self-paced graph contrastive learning.
\newblock In \emph{SIGKDD}, pp.\ 936--944, 2022.
\bibitem[Liu et~al.(2023{\natexlab{a}})Liu, Yuan, Fu, Jiang, Hayashi, and Neubig]{liu2023pre}
Pengfei Liu, Weizhe Yuan, Jinlan Fu, Zhengbao Jiang, Hiroaki Hayashi, and Graham Neubig.
\newblock Pre-train, prompt, and predict: A systematic survey of prompting methods in natural language processing.
\newblock \emph{ACM Computing Surveys}, 55\penalty0 (9), 2023{\natexlab{a}}.
\bibitem[Liu et~al.(2023{\natexlab{b}})Liu, Yu, Fang, and Zhang]{liu2023graphprompt}
Zemin Liu, Xingtong Yu, Yuan Fang, and Xinming Zhang.
\newblock {GraphPrompt}: Unifying pre-training and downstream tasks for graph neural networks.
\newblock In \emph{WWW}, pp.\ 417--428, 2023{\natexlab{b}}.
\bibitem[Liyanagunawardena et~al.(2013)Liyanagunawardena, Adams, and Williams]{liyanagunawardena2013moocs}
Tharindu~Rekha Liyanagunawardena, Andrew~Alexandar Adams, and Shirley~Ann Williams.
\newblock {MOOCs}: A systematic study of the published literature 2008-2012.
\newblock \emph{International Review of Research in Open and Distributed Learning}, 14\penalty0 (3):\penalty0 202--227, 2013.
\bibitem[Nguyen et~al.(2018)Nguyen, Lee, Rossi, Ahmed, Koh, and Kim]{nguyen2018continuous}
Giang~Hoang Nguyen, John~Boaz Lee, Ryan~A Rossi, Nesreen~K Ahmed, Eunyee Koh, and Sungchul Kim.
\newblock Continuous-time dynamic network embeddings.
\newblock In \emph{WWW Companion}, pp.\ 969--976, 2018.
\bibitem[Pan et~al.(2019)Pan, Li, Ao, Tang, and He]{pan2019warm}
Feiyang Pan, Shuokai Li, Xiang Ao, Pingzhong Tang, and Qing He.
\newblock Warm up cold-start advertisements: Improving ctr predictions via learning to learn id embeddings.
\newblock In \emph{SIGIR}, pp.\ 695--704, 2019.
\bibitem[Pareja et~al.(2020)Pareja, Domeniconi, Chen, Ma, Suzumura, Kanezashi, Kaler, Schardl, and Leiserson]{pareja2020evolvegcn}
Aldo Pareja, Giacomo Domeniconi, Jie Chen, Tengfei Ma, Toyotaro Suzumura, Hiroki Kanezashi, Tim Kaler, Tao Schardl, and Charles Leiserson.
\newblock {EvolveGCN}: Evolving graph convolutional networks for dynamic graphs.
\newblock In \emph{AAAI}, pp.\ 5363--5370, 2020.
\bibitem[Qu et~al.(2023)Qu, Tang, Zheng, Nguyen, Huang, Shi, and Yin]{qu2023semi}
Liang Qu, Ningzhi Tang, Ruiqi Zheng, Quoc Viet~Hung Nguyen, Zi~Huang, Yuhui Shi, and Hongzhi Yin.
\newblock Semi-decentralized federated ego graph learning for recommendation.
\newblock In \emph{WWW}, pp.\ 339--348, 2023.
\bibitem[Rossi et~al.(2020)Rossi, Chamberlain, Frasca, Eynard, Monti, and Bronstein]{rossi2020temporal}
Emanuele Rossi, Ben Chamberlain, Fabrizio Frasca, Davide Eynard, Federico Monti, and Michael Bronstein.
\newblock Temporal graph networks for deep learning on dynamic graphs.
\newblock \emph{arXiv preprint arXiv:2006.10637}, 2020.
\bibitem[Skarding et~al.(2021)Skarding, Gabrys, and Musial]{skarding2021foundations}
Joakim Skarding, Bogdan Gabrys, and Katarzyna Musial.
\newblock Foundations and modeling of dynamic networks using dynamic graph neural networks: A survey.
\newblock \emph{IEEE Access}, 9:\penalty0 79143--79168, 2021.
\bibitem[Sun et~al.(2022{\natexlab{a}})Sun, Ye, Peng, and Yu]{sun2022self}
Li~Sun, Junda Ye, Hao Peng, and Philip~S Yu.
\newblock A self-supervised {Riemannian GNN} with time varying curvature for temporal graph learning.
\newblock In \emph{CIKM}, pp.\ 1827--1836, 2022{\natexlab{a}}.
\bibitem[Sun et~al.(2022{\natexlab{b}})Sun, Zhou, He, Wang, and Wang]{sun2022gppt}
Mingchen Sun, Kaixiong Zhou, Xin He, Ying Wang, and Xin Wang.
\newblock {GPPT}: Graph pre-training and prompt tuning to generalize graph neural networks.
\newblock In \emph{SIGKDD}, pp.\ 1717--1727, 2022{\natexlab{b}}.
\bibitem[Sun et~al.(2023)Sun, Cheng, Li, Liu, and Guan]{sun2023all}
Xiangguo Sun, Hong Cheng, Jia Li, Bo~Liu, and Jihong Guan.
\newblock All in one: Multi-task prompting for graph neural networks.
\newblock \emph{SIGKDD}, pp.\ 2120--2131, 2023.
\bibitem[Tan et~al.(2023)Tan, Guo, Ding, and Liu]{tan2023virtual}
Zhen Tan, Ruocheng Guo, Kaize Ding, and Huan Liu.
\newblock Virtual node tuning for few-shot node classification.
\newblock In \emph{SIGKDD}, pp.\ 2177--2188, 2023.
\bibitem[Tian et~al.(2021)Tian, Wu, Shi, Zhu, and Xiong]{tian2021self}
Sheng Tian, Ruofan Wu, Leilei Shi, Liang Zhu, and Tao Xiong.
\newblock Self-supervised representation learning on dynamic graphs.
\newblock In \emph{CIKM}, pp.\ 1814--1823, 2021.
\bibitem[Trivedi et~al.(2019)Trivedi, Farajtabar, Biswal, and Zha]{trivedi2019dyrep}
Rakshit Trivedi, Mehrdad Farajtabar, Prasenjeet Biswal, and Hongyuan Zha.
\newblock {DyRep}: Learning representations over dynamic graphs.
\newblock In \emph{ICLR}, 2019.
\bibitem[Veli{\v{c}}kovi{\'c} et~al.(2018)Veli{\v{c}}kovi{\'c}, Cucurull, Casanova, Romero, Lio, and Bengio]{velivckovic2017graph}
Petar Veli{\v{c}}kovi{\'c}, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio.
\newblock Graph attention networks.
\newblock In \emph{ICLR}, 2018.
\bibitem[Wang et~al.(2023)Wang, Liang, Li, Li, Ghanem, Zimmermann, Zhou, Yi, Zhang, and Wang]{wang2023brave}
Kun Wang, Yuxuan Liang, Xinglin Li, Guohao Li, Bernard Ghanem, Roger Zimmermann, Zhengyang Zhou, Huahui Yi, Yudong Zhang, and Yang Wang.
\newblock Brave the wind and the waves: Discovering robust and generalizable graph lottery tickets.
\newblock \emph{IEEE TPAMI}, 46\penalty0 (5):\penalty0 3388--3405, 2023.
\bibitem[Wang et~al.(2021)Wang, Chang, Liu, Leskovec, and Li]{wang2021inductive}
Yanbang Wang, Yen-Yu Chang, Yunyu Liu, Jure Leskovec, and Pan Li.
\newblock Inductive representation learning in temporal networks via causal anonymous walks.
\newblock In \emph{ICLR}, 2021.
\bibitem[Wen \& Fang(2022)Wen and Fang]{wen2022trend}
Zhihao Wen and Yuan Fang.
\newblock {TREND}: Tempo{R}al event and node dynamics for graph representation learning.
\newblock In \emph{WWW}, pp.\ 1159--1169, 2022.
\bibitem[Wen \& Fang(2023)Wen and Fang]{wen2023prompt}
Zhihao Wen and Yuan Fang.
\newblock Prompt tuning on graph-augmented low-resource text classification.
\newblock \emph{IEEE TKDE}, 36\penalty0 (12):\penalty0 9080--9095, 2023.
\bibitem[Wu \& Lee(2018)Wu and Lee]{wu2018reducing}
Yuzhong Wu and Tan Lee.
\newblock Reducing model complexity for dnn based large-scale audio classification.
\newblock In \emph{ICASSP}, pp.\ 331--335, 2018.
\bibitem[Wu et~al.(2020)Wu, Pan, Chen, Long, Zhang, and Philip]{wu2020comprehensive}
Zonghan Wu, Shirui Pan, Fengwen Chen, Guodong Long, Chengqi Zhang, and S~Yu Philip.
\newblock A comprehensive survey on graph neural networks.
\newblock \emph{IEEE TNNLS}, 32\penalty0 (1):\penalty0 4--24, 2020.
\bibitem[Xu et~al.(2020)Xu, Ruan, Korpeoglu, Kumar, and Achan]{xu2020inductive}
Da~Xu, Chuanwei Ruan, Evren Korpeoglu, Sushant Kumar, and Kannan Achan.
\newblock Inductive representation learning on temporal graphs.
\newblock In \emph{ICLR}, 2020.
\bibitem[Xu et~al.(2022)Xu, Wu, Liu, Wu, and Wang]{xu2022evidence}
Weizhi Xu, Junfei Wu, Qiang Liu, Shu Wu, and Liang Wang.
\newblock Evidence-aware fake news detection with graph neural networks.
\newblock In \emph{WWW}, pp.\ 2501--2510, 2022.
\bibitem[Yao et~al.(2020)Yao, Zhang, Wei, Jiang, Wang, Huang, Chawla, and Li]{yao2020graph}
Huaxiu Yao, Chuxu Zhang, Ying Wei, Meng Jiang, Suhang Wang, Junzhou Huang, Nitesh Chawla, and Zhenhui Li.
\newblock Graph few-shot learning via knowledge transfer.
\newblock In \emph{AAAI}, pp.\ 6656--6663, 2020.
\bibitem[You et~al.(2022)You, Du, and Leskovec]{you2022roland}
Jiaxuan You, Tianyu Du, and Jure Leskovec.
\newblock {ROLAND}: Graph learning framework for dynamic graphs.
\newblock In \emph{SIGKDD}, pp.\ 2358--2366, 2022.
\bibitem[You et~al.(2020)You, Chen, Sui, Chen, Wang, and Shen]{you2020graph}
Yuning You, Tianlong Chen, Yongduo Sui, Ting Chen, Zhangyang Wang, and Yang Shen.
\newblock Graph contrastive learning with augmentations.
\newblock In \emph{NeurIPS}, pp.\ 5812--5823, 2020.
\bibitem[Yu et~al.(2023{\natexlab{a}})Yu, Sun, Du, and Lv]{yu2023towards}
Le~Yu, Leilei Sun, Bowen Du, and Weifeng Lv.
\newblock Towards better dynamic graph learning: New architecture and unified library.
\newblock In \emph{NeurIPS}, pp.\ 67686--67700, 2023{\natexlab{a}}.
\bibitem[Yu et~al.(2023{\natexlab{b}})Yu, Liu, Fang, and Zhang]{yu2023hgprompt}
Xingtong Yu, Zemin Liu, Yuan Fang, and Xinming Zhang.
\newblock {HGPrompt}: Bridging homogeneous and heterogeneous graphs for few-shot prompt learning.
\newblock In \emph{AAAI}, pp.\ 16578--16586, 2023{\natexlab{b}}.
\bibitem[Yu et~al.(2023{\natexlab{c}})Yu, Liu, Fang, and Zhang]{yu2023learning}
Xingtong Yu, Zemin Liu, Yuan Fang, and Xinming Zhang.
\newblock Learning to count isomorphisms with graph neural networks.
\newblock In \emph{AAAI}, pp.\ 4845--4853, 2023{\natexlab{c}}.
\bibitem[Yu et~al.(2023{\natexlab{d}})Yu, Liu, Fang, Liu, Chen, and Zhang]{yu2023generalized}
Xingtong Yu, Zhenghao Liu, Yuan Fang, Zemin Liu, Sihong Chen, and Xinming Zhang.
\newblock Generalized graph prompt: Toward a unification of pre-training and downstream tasks on graphs.
\newblock \emph{IEEE TKDE}, 36\penalty0 (11):\penalty0 6237-- 6250, 2023{\natexlab{d}}.
\bibitem[Yu et~al.(2023{\natexlab{e}})Yu, Zhou, Fang, and Zhang]{yu2023multigprompt}
Xingtong Yu, Chang Zhou, Yuan Fang, and Xinming Zhang.
\newblock {MultiGPrompt} for multi-task pre-training and prompting on graphs.
\newblock In \emph{WWW}, pp.\ 515--526, 2023{\natexlab{e}}.
\bibitem[Yu et~al.(2024{\natexlab{a}})Yu, Fang, Liu, Wu, Wen, Bo, Zhang, and Hoi]{yu2024few}
Xingtong Yu, Yuan Fang, Zemin Liu, Yuxia Wu, Zhihao Wen, Jianyuan Bo, Xinming Zhang, and Steven~CH Hoi.
\newblock Few-shot learning on graphs: from meta-learning to pre-training and prompting.
\newblock \emph{arXiv preprint arXiv:2402.01440}, 2024{\natexlab{a}}.
\bibitem[Yu et~al.(2024{\natexlab{b}})Yu, Zhou, Fang, and Zhang]{yu2024text}
Xingtong Yu, Chang Zhou, Yuan Fang, and Xinming Zhang.
\newblock Text-free multi-domain graph pre-training: Toward graph foundation models.
\newblock \emph{arXiv preprint arXiv:2405.13934}, 2024{\natexlab{b}}.
\bibitem[Yu et~al.(2025{\natexlab{a}})Yu, Gong, Chang, Fang, and Hui]{yu2025samgpt}
Xingtong Yu, Zechuan Gong, Zhou Chang, Yuan Fang, and Zhang Hui.
\newblock {SAMGPT}: Text-free graph foundation model for multi-domain pre-training and cross-domain adaptation.
\newblock In \emph{WWW}, 2025{\natexlab{a}}.
\bibitem[Yu et~al.(2025{\natexlab{b}})Yu, Zhang, Fang, and Jiang]{yu2024non}
Xingtong Yu, Jie Zhang, Yuan Fang, and Renhe Jiang.
\newblock Non-homophilic graph pre-training and prompt learning.
\newblock In \emph{SIGKDD}, 2025{\natexlab{b}}.
\bibitem[Yu et~al.(2025{\natexlab{c}})Yu, Zhou, Kuai, Zhang, and Fang]{yu2025gcot}
Xingtong Yu, Chang Zhou, Zhongwei Kuai, Xinming Zhang, and Yuan Fang.
\newblock {GCoT}: Chain-of-thought prompt learning for graphs.
\newblock \emph{arXiv preprint arXiv:2502.08092}, 2025{\natexlab{c}}.
\bibitem[Zhang et~al.(2024)Zhang, Yue, Sun, Wan, Yu, Fang, Wang, and Cheng]{zhang2024g}
Guibin Zhang, Yanwei Yue, Xiangguo Sun, Guancheng Wan, Miao Yu, Junfeng Fang, Kun Wang, and Dawei Cheng.
\newblock {G-Designer}: Architecting multi-agent communication topologies via graph neural networks.
\newblock \emph{arXiv preprint arXiv:2410.11782}, 2024.
\bibitem[Zhou et~al.(2019)Zhou, Cao, Zhang, Trajcevski, Zhong, and Geng]{zhou2019meta}
Fan Zhou, Chengtai Cao, Kunpeng Zhang, Goce Trajcevski, Ting Zhong, and Ji~Geng.
\newblock {Meta-GNN}: On few-shot node classification in graph meta-learning.
\newblock In \emph{CIKM}, pp.\ 2357--2360, 2019.
\bibitem[Zhou et~al.(2022)Zhou, Yang, Loy, and Liu]{zhou2022conditional}
Kaiyang Zhou, Jingkang Yang, Chen~Change Loy, and Ziwei Liu.
\newblock Conditional prompt learning for vision-language models.
\newblock In \emph{CVPR}, pp.\ 16816--16825, 2022.
\end{thebibliography}