Complex Physics-Informed Neural Network
Extracted source text · arXiv 2502.04917
\documentclass{article}
\usepackage{amsthm}
\newtheorem{theorem}{Theorem}
\usepackage{arxiv}
%\usepackage{amsthm} % This package helps define theorem-like environments
% Define a new "Remark" environment
%\newtheorem{remark}{Remark}[section]
\usepackage[utf8]{inputenc} % allow utf-8 input
\usepackage[T1]{fontenc} % use 8-bit T1 fonts
\usepackage{hyperref} % hyperlinks
\usepackage{url}
%\usepackage{amsthm}
\usepackage{amssymb}
\usepackage{comment}
\usepackage{latexsym}
\usepackage{amsmath}
\usepackage{enumitem}
\usepackage{cite}
\usepackage{multirow}
\usepackage{bm}
\usepackage{makecell}
% simple URL typesetting
\usepackage{booktabs} % professional-quality tables
\usepackage{amsfonts} % blackboard math symbols
\usepackage{nicefrac} % compact symbols for 1/2, etc.
\usepackage{microtype} % microtypography
\usepackage{lipsum}
\usepackage{graphicx}
\usepackage{subcaption}
\usepackage{xcolor}
\usepackage{algorithm}
\usepackage{algorithmic}
\graphicspath{ {./images/} }
\newcommand{\nn}{\nonumber}
\newcommand{\be}{\begin{eqnarray}}
\newcommand{\ee}{\end{eqnarray}}
\newcommand{\mc}[1]{\mathcal{#1}}
\newcommand{\mf}[1]{\mathbf{#1}}
\newcommand{\mb}[1]{\mathbb{#1}}
\newcommand{\my}[1]{{\color{red} #1}}
\usepackage{amsfonts} % For \mathbb{C}
\title{Complex Physics-Informed Neural Network}
\author{
Chenhao Si \\
School of Data Science\\
The Chinese University of Hong Kong, Shenzhen\\
Shenzhen, China \\
\texttt{222042011@link.cuhk.edu.cn} \\
%% examples of more authors
\And
Ming Yan \\
School of Data Science\\
The Chinese University of Hong Kong, Shenzhen \\
Shenzhen, China \\
\texttt{yanming@cuhk.edu.cn} \\
\And
Xin Li\\
Department of Computer Science\\
Northwestern University\\
IL, USA \\
\texttt{xinli2023@u.northwestern.edu} \\
\And
Zhihong Xia* \\School of Science, Great Bay University \\ Guangdong, China \\
\& Department of Mathematics\\
Northwestern University\\
IL, USA \\
\texttt{xia@math.northwestern.edu} \\
%% \AND
%% Coauthor \\
%% Affiliation \\
%% Address \\
%% \texttt{email} \\
%% \And
%% Coauthor \\
%% Affiliation \\
%% Address \\
%% \texttt{email} \\
%% \And
%% Coauthor \\
%% Affiliation \\
%% Address \\
%% \texttt{email} \\
}
\begin{document}
\maketitle
\begin{abstract}
We propose compleX-PINN, a novel physics-informed neural network (PINN) architecture incorporating a learnable activation function inspired by Cauchy’s integral theorem. By optimizing the activation parameters, compleX-PINN achieves high accuracy with just a single hidden layer. Empirically, we demonstrate that compleX-PINN solves high-dimensional problems that pose significant challenges for PINNs. Our results show compleX-PINN consistently achieves substantially greater precision, often improving accuracy by an order of magnitude, on these complex tasks.
\end{abstract}
% keywords can be removed
%\keywords{First keyword \and Second keyword \and More}
\section{Introduction}
Physics-Informed Neural Networks (PINNs) have emerged as a powerful method for solving both forward and inverse problems involving Partial Differential Equations (PDEs)~\cite{ref1, ref2, ref3, ref4}.
PINNs leverage the expressive power of neural networks to minimize a loss function that enforces the governing PDEs and boundary/initial conditions. This approach has been widely applied across various domains, including heat transfer~\cite{ref5, ref6, ref7}, solid mechanics~\cite{ref8, ref9, ref10}, incompressible flows~\cite{ref11, ref12, ref13}, stochastic differential equations~\cite{ref14, ref15}, and uncertainty quantification~\cite{ref16, ref17}.
Despite their success, PINNs face significant challenges and often struggle to solve certain classes of problems~\cite{ref18, ref19}. One major difficulty arises in scenarios where the solution exhibits rapid changes, such as in ‘stiff’ PDEs~\cite{ref20}, leading to issues with convergence and accuracy. To address these limitations, researchers have proposed various techniques to improve training efficiency and precision.
Over the years, numerous strategies have been developed to enhance the performance of PINNs, including adaptive weighting of loss functions and selective sampling of training points. For example, Wang et al.\cite{ref19} leveraged the Neural Tangent Kernel (NTK) to analyze gradient evolution in PINN training, adjusting the weights of each loss component accordingly. Other studies\cite{ref21, ref22} have explored methods for dynamically learning these weights during training. Additionally, adaptive sampling techniques have been introduced to tackle stiff problems by focusing on regions with high residuals. Lu et al.\cite{ref23} proposed a threshold-based approach for selecting new training points, while Wu et al.\cite{ref24} introduced a probability density function derived from residuals to improve sampling efficiency. Further extensions include active learning-based sampling methods~\cite{gao2023active} and re-sampling techniques targeting failure regions~\cite{ref26}.
Beyond these approaches, causality has been recognized as an influential factor in PINN training~\cite{ref27, RRef3}. Wang et al.~\cite{ref27} introduced a Causality-PINN, which assigns time-dependent weights to the loss function for time-dependent PDEs, while Daw et al.\cite{RRef3} further integrated causality with importance sampling techniques.
Recent advancements in Physics-Informed Neural Networks (PINNs) include domain decomposition methods~\cite{ref28, ref29, ref30, ref31}, improved initialization schemes~\cite{ref31, ref32, ref33}, novel loss functions~\cite{RRef5}, and innovative network architectures. For example, Kolmogorov-Arnold Networks (KANs)~\cite{liu2024kan}—originally designed for general function representation—employ spline-based activation functions inspired by the Kolmogorov-Arnold theorem to enhance expressivity. Here, we adapt KANs for solving partial differential equations (PDEs), termed PIKAN~\cite{shukla2024comprehensive,wang2025kolmogorov}, to leverage their adaptive basis functions while addressing challenges in high-dimensional scalability. Alongside other architectural innovations like hierarchical networks~\cite{RRef2, PIRBN, ref34, ref35,wang2024piratenets}, such advancements—including our PDE-focused adaptation of KANs—demonstrate the field’s progress in tailoring neural networks to the structure of physical systems. Collectively, these improvements have expanded PINNs’ utility for complex real-world applications, though optimizing expressivity and efficiency in high dimensions remains critical.
However, existing methods primarily rely on additional training techniques or significantly larger models, such as transformers~\cite{RRef2} and convolutional neural networks~\cite{ref35}. While these approaches can enhance performance, they also introduce substantial computational overhead. Auxiliary networks and gradient-based modifications~\cite{ref36, ref37} increase training costs, whereas larger models suffer from slower convergence due to the large number of parameters~\cite{ref18, RRef4}. Even recent innovations like KANs, despite their theoretical promise, face scalability challenges in high-dimensional PDE settings, as shown in our experiments (Section~\ref{Section Poisson}).
To overcome these limitations, developing a network that enhances both convergence and predictive accuracy without relying on additional training algorithms or significantly increasing model complexity is crucial. Motivated by this, we introduce compleX-PINN, which incorporates the Cauchy integral formula~\cite{XNet1, XNet2} into a novel activation function, offering a more efficient and effective alternative to traditional PINNs.
The main contributions of this paper are summarized as follows:
\begin{itemize}
\item To the best of our knowledge, this is the first comprehensive study on using Cauchy-based activation functions in PINNs.
\item We provide a detailed derivation and motivation for incorporating Cauchy activation functions into PINN architectures.
\item Empirical results demonstrate that compleX-PINN outperforms several PINN-based models.
\item We show that compleX-PINN is compatible with existing PINN training techniques, further enhancing its performance when integrated with these methods.
\end{itemize}
The organization of this paper is as follows. Section~\ref{Sec2.1} provides a brief introduction to PINNs. Our proposed model, compleX-PINN, is introduced in Section~\ref{Cauchy activation function}, where we first present the Cauchy activation function using Cauchy's 1D integral formula in Section~\ref{Sec 3.1}, extend it to high-dimensional cases in Section~\ref{Sec 3.2}, and apply it to neural networks in Section~\ref{Extend to high dimension}.
Finally, numerical results for testing several PDEs in high dimension are presented in Section~\ref{Sec4}, followed by the conclusion.
\section{Physics-Informed Neural Network}
\label{Sec2.1}
Denote the spatial domain as $\Omega \subset \mathbb{R}^n$ with boundary $\partial \Omega$, and let $T$ represent the time domain. The spatial-temporal variable is given by $(\mathbf{x}, t) \in \Omega \times T$. A time-dependent partial differential equation (PDE) over this domain is defined as follows:
\begin{align}
\mathcal{F}[u](\mathbf{x}, t) &= 0, \label{(1)} \quad (\mathbf{x}, t) \in \Omega \times T, \\
\mathcal{B}[u](\mathbf{x}, t) &= 0, \label{(2)} \quad (\mathbf{x}, t) \in \partial \Omega \times T, \quad \text{(boundary condition)} \\
\mathcal{I}[u](\mathbf{x}, 0) &= 0, \quad \mathbf{x} \in \Omega, \hspace{51pt} \text{(initial condition)}
\end{align}
where $\mathcal{F}$, $\mathcal{B}$, and $\mathcal{I}$ are differential operators, and $u(\mathbf{x}, t)$ is the solution to the PDE, subject to boundary and initial conditions.
A PINN parameterized by $\theta$ approximates the solution $u(\mathbf{x}, t)$. The input to the neural network is $(\mathbf{x}, t)$, and the approximation is denoted by $\hat{u}(\theta)(\mathbf{x}, t)$. The PINN minimizes the following objective function:
\begin{align}
\mathcal{L}(\theta) = \lambda_F \mathcal{L}_F(\theta) + \lambda_B \mathcal{L}_B(\theta) + \lambda_I \mathcal{L}_I(\theta), \label{(3)}
\end{align}
where
\begin{align}
\mathcal{L}_F(\theta) &= \frac{1}{N_f} \sum_{(\mathbf{x}, t) \in \Omega_F} \big| \mathcal{F}[\hat{u}(\theta)](\mathbf{x}, t) \big|^2, \label{(4)} \\
\mathcal{L}_B(\theta) &= \frac{1}{N_b} \sum_{(\mathbf{x}, t) \in \Omega_B} \big| \mathcal{B}[\hat{u}(\theta)](\mathbf{x}, t) \big|^2, \label{(5)} \\
\mathcal{L}_I(\theta) &= \frac{1}{N_0} \sum_{(\mathbf{x}, 0) \in \Omega_I} \big| \mathcal{I}[\hat{u}(\theta)](\mathbf{x}, 0) \big|^2. \label{(6)}
\end{align}
Here, $\Omega_F$, $\Omega_B$, and $\Omega_I$ are the training sets for the PDE residual, boundary condition, and initial condition, respectively, with cardinalities $N_f$, $N_b$, and $N_0$. The weights $\lambda_F$, $\lambda_B$, and $\lambda_I$ are hyper-parameters tuning the contributions of each loss component. Notably, $\Omega_F$ may include points on the boundary or at the initial time, allowing $\Omega_F \cap \Omega_B$ and $\Omega_F \cap \Omega_I$ to be non-empty.
The choice of activation function $\sigma(\cdot)$ is crucial in PINNs, as it introduces the nonlinearity necessary to approximate complex solutions to PDEs. The hyperbolic tangent (tanh) is commonly used in PINNs for its smoothness and training stability~\cite{ref1, ref2}. Wavelet-based activations have also been explored to capture multi-scale features~\cite{ref40, RRef2, ref34}. Recent work by Li et al. \cite{XNet1} proposed the Cauchy activation function, which has shown strong performance in computer vision and time-series forecasting tasks \cite{XNet2}.
In this study, we extend the application of the Cauchy activation function to address PDEs that are challenging for standard PINNs. The Cauchy activation function and the novel compleX-PINN model are introduced in the following section.
\section{Complex Physics-Informed Neural Network}
\label{Cauchy activation function}
The Cauchy activation function, introduced in \cite{XNet1}, is defined as:
\begin{align}
\Phi(x; \mu_1, \mu_2, d) = \frac{\mu_1 x}{x^2 + d^2} + \frac{\mu_2}{x^2 + d^2}, \label{(12)}
\end{align}
where $\mu_1$, $\mu_2$, and $d$ are trainable parameters. Moreover, $\mu_1$ governs the linear component of the input $x$, $\mu_2$ regulates the constant, enhancing adaptability in shaping the activation, and $d$ controls the activation function’s range and smoothness properties. The initialization effect of each parameters will be discussed later in Section~\ref{Sec Robustness}. This activation function is inspired by Cauchy's integral formula, as we further elaborate in Section~\ref{Sec 3.1}. We refer to a PINN model employing the Cauchy activation function as compleX-PINN.
We would like to note at the outset that our network is initially constructed with a single hidden layer, where each neuron has a unique set of trainable parameters $\{\mu_1,\mu_2,d\}$. Consequently, the total number of trainable parameters for the Cauchy activation function is $3\times N_{\text{Cauchy}}$, where $N_{\text{Cauchy}}$ represents the width of the layer.
\subsection{1D Cauchy's integral formula and the Cauchy activation function}
\label{Sec 3.1}
This section introduces Cauchy's integral formula and derives the Cauchy activation function from it.
\begin{theorem}[Cauchy's Integral Formula]
\label{theorem 1}
Let $f$ be a complex-valued function on the complex plane. If $f$ is holomorphic inside and on a simple closed curve \( C \), and $z$ is a point inside \( C \), then:
\[
f(z) = \frac{1}{2\pi i} \oint_C \frac{f(\zeta)}{\zeta - z} \, d\zeta.
\]
\end{theorem}
Cauchy's integral formula expresses the value of a function at any point $z$ as a function of known values along a closed curve \( C \) that encloses \( z \).
Remarkably, this principle is akin to machine learning, where the values at new points are inferred from the known values.
In practice, we approximate the integral using a Riemann sum over a finite number of points on the curve \( C \). Let \( \zeta^1, \zeta^2, \ldots, \zeta^m \) be a sequence of \( m \) points on \( C \). Then,
\begin{align}
f(z) \approx \frac{1}{2\pi i} \sum_{k=1}^{m} \frac{f(\zeta^{k})}{\zeta^{k} - z} \, (\zeta^{k+1} - \zeta^{k}) := \sum_{k=1}^m \frac{\lambda_k}{\zeta^k - z}, \label{equ9}
\end{align}
where, for convenience, we set \( \zeta^{m+1} = \zeta^1 \) and define \( \lambda_k = \frac{f(\zeta^k) \, (\zeta^{k+1} - \zeta^k)}{2\pi i} \).
If our target function \( f \) is real and one-dimensional, we obtain:
\begin{align}
f(x) \approx \text{Re} \left( \sum_{k=1}^m \frac{\lambda_k}{\zeta^k - x} \right) = \sum_{k=1}^m \frac{\text{Re}(\lambda_k) \, \text{Re}(\zeta^k) + \text{Im}(\lambda_k) \, \text{Im}(\zeta^k) - \text{Re}(\lambda_k) x}{(x - \text{Re}(\zeta^k))^2 + (\text{Im}(\zeta^k))^2}. \label{(11)}
\end{align}
With the Cauchy activation function defined in~\eqref{(12)}, we have
\begin{align}
f(x) \approx \sum_{k=1}^m \Phi \left( x - \text{Re}(\zeta^k); -\text{Re}(\lambda_k), \, \text{Re}(\lambda_k) \, \text{Re}(\zeta^k) + \text{Im}(\lambda_k) \, \text{Im}(\zeta^k), \, (\text{Im}(\zeta^k))^2 \right).
\end{align}
This shows that a one-layer neural network with the Cauchy activation function~\eqref{(12)} can approximate the real function \( f(x) \).
%At this stage, we highlight the distinctive advantage of the Cauchy-based approximation embedded within compleX-PINN. Traditional activation functions—such as ReLU or Sigmoid—typically possess, at most, first-order approximation capabilities. This inherently limits their expressiveness in modeling complex solution behaviors. In contrast, the Cauchy approximation offers a natural pathway to higher-order accuracy, with its error explicitly arising from the discretization of a contour integral.
At this stage, we emphasize the unique strength of the Cauchy activation function embedded in compleX-PINN. Unlike traditional activation functions, such as ReLU or Sigmoid, which typically offer only first-order approximation capabilities and are thus limited in expressiveness. The Cauchy activation function provides a natural path to high-order and even exponential approximation accuracy.
The key idea is that the activation function is derived from a discretized version of the Cauchy integral formula, where the network learns both the sample points and the associated weights. For analytic functions defined on compact domains, the approximation error of the Cauchy integral formula decays as $O(r^{-m})$ for some $r < 1$, where $m$ is the number of sampling points and $r$ depends on the analyticity properties of the function and the geometry of its domain~\cite{trefethen2014exponentially}. This exponential convergence is significantly faster than the algebraic decay $O(m^{-p})$ achieved by polynomial-based approximations of order $p$. In other words, the Cauchy activation function enables more efficient and accurate function representation, particularly for smooth or analytic functions.
While various numerical integration schemes—such as the trapezoidal rule, Simpson’s rule, Boole’s rule, or more general Newton–Cotes formulas—can be used to discretize the Cauchy integral, they ultimately produce the same functional form for the activation, up to the specific values of the quadrature weights $\lambda_k$ and nodes $z_k$. Since these quantities are treated as trainable parameters in the neural network, the choice of integration rule becomes inconsequential. The network adapts them to best approximate the target function during training.
% Although one might consider employing higher-order Newton–Cotes rules (e.g., Simpson’s or Boole’s rule) for the numerical integration, this would ultimately yield the same Cauchy-based approximation formula~(\ref{equ9}). The only difference lies in the values of the weights $\lambda_k$, which would vary slightly depending on the chosen integration scheme. However, this variation is negligible in practice, as $\lambda_k$ are trainable parameters optimized to fit the data.
% Ultimately, the order of approximation can be arbitrarily high. In fact, for any {\color{blue}positive integer $p$ (representing the desired order of approximation), the error term is estimated to be of the order $o(n^{-p})$, meaning the error decays faster than any polynomial function of $n$~\cite{XNet1}.}
% {\color{blue}For analytic functions on compact domains, the pointwise approximation error $\epsilon$ decays faster than any polynomial order: given any integer $p>0$, there exists a constant $C(p)$ (depending on $p$ and the function) such that $\epsilon \leq C(p) m^{-p}$ uniformly for all $x$ in the domain. $C(p)$ typically increases with $p$, so very high-order decay manifests only for sufficiently large $m$.}
% %The Cauchy activation function has many advantages over the traditional functions that PINN often uses as we will show.
\subsection{Multi-dimensional Cauchy's integral formula}
\label{Sec 3.2}
This section extends Cauchy's integral formula to the multi-dimensional case.
\begin{theorem}[Multi-Dimensional Cauchy's Integral Formula]
\label{theorem 2}
Let \( f(z) \) be holomorphic in a compact domain \( U \subset \mathbb{C}^N \) within \( N \)-dimensional complex space. For simplicity, assume that \( U \) has a product structure: \( U = U_1 \times U_2 \times \ldots \times U_N \), where each \( U_i \), \( i = 1, 2, \dots, N \), is a compact domain in the complex plane. Let \( P \) denote the surface defined by
\[
P = \partial U_1 \times \partial U_2 \times \ldots \times \partial U_N,
\]
then a multi-dimensional extension of Cauchy's integral formula for \( (z_1, z_2, \ldots, z_N) \in U \) is given by:
\[
f(z_1, z_2, \ldots, z_N) = \left(\frac{1}{2\pi i}\right)^N \int\cdots\int_P \frac{f(\zeta_1, \zeta_2, \ldots, \zeta_N)}{(\zeta_1 - z_1)(\zeta_2 - z_2) \cdots (\zeta_N - z_N)} \, d\zeta_1 \cdots d\zeta_N.
\]
\end{theorem}
Similarly, we approximate the integral by a Riemann sum over a finite number of points. More precisely, for any integer \( l = 1, \ldots, N \), let
\( \zeta_l^1, \zeta_l^2, \ldots, \zeta_l^{m_l} \) be a sequence of \( m_l \) points on \( \partial U_l \). Then,
\begin{align}
f(z_1, z_2, \ldots, z_N) \approx \left(\frac{1}{2\pi i}\right)^N
\sum_{k_1=1}^{m_1} \cdots \sum_{k_N=1}^{m_N}
\frac{f(\zeta_1^{k_1}, \zeta_2^{k_2}, \ldots, \zeta_N^{k_N})}{(\zeta_1^{k_1} - z_1)(\zeta_2^{k_2} - z_2) \cdots (\zeta_N^{k_N} - z_N)}
(\zeta_1^{k_1+1} - \zeta_1^{k_1}) \cdots (\zeta_N^{k_N+1} - \zeta_N^{k_N}),
\end{align}
where, for convenience, we set \( \zeta_l^{m_l+1} = \zeta_l^1 \) for \( l = 1, 2, \ldots, N \).
Collecting all terms that are independent of \( z_1, \ldots, z_N \), we define
\[
\lambda_{k_1, \ldots, k_N} = \left(\frac{1}{2\pi i}\right)^N f(\zeta_1^{k_1}, \zeta_2^{k_2}, \ldots, \zeta_N^{k_N}) (\zeta_1^{k_1+1} - \zeta_1^{k_1}) \cdots (\zeta_N^{k_N+1} - \zeta_N^{k_N}),
\]
so that we can rewrite the approximation as
\begin{align}
f(z_1, z_2, \ldots, z_N) \approx \sum_{k_1=1}^{m_1} \cdots \sum_{k_N=1}^{m_N} \frac{\lambda_{k_1, \ldots, k_N}}{(\zeta_1^{k_1} - z_1)(\zeta_2^{k_2} - z_2) \cdots (\zeta_N^{k_N} - z_N)}.
\end{align}
Since the order of the sample points no longer matters, we can rewrite the sample points as a single sequence \( (\zeta_1^k, \ldots, \zeta_N^k) \) for \( k = 1, 2, \ldots, m \), where \( m = m_1 m_2 \cdots m_N \). Thus, we finally obtain
\begin{align}
f(z_1, z_2, \ldots, z_N) \approx \sum_{k=1}^m \frac{\lambda_k}{(\zeta_1^k - z_1)(\zeta_2^k - z_2) \cdots (\zeta_N^k - z_N)}, \label{(8)}
\end{align}
where \( \lambda_1, \lambda_2, \ldots, \lambda_m \) are parameters that depend on the sample points \( (\zeta_1^k, \zeta_2^k, \ldots, \zeta_N^k) \) and the values \( f(\zeta_1^k, \zeta_2^k, \ldots, \zeta_N^k) \) for \( k = 1, 2, \ldots, m \).
While the multi-dimensional Cauchy integral formula provides a powerful theoretical foundation for approximating holomorphic functions, its direct application in high-dimensional settings poses computational challenges. Specifically, the number of terms in the approximation grows exponentially with dimension $N$, and the resulting structure can be difficult to implement and optimize efficiently within standard neural network architectures. To address this, we propose a more practical alternative in the next section: approximating high-dimensional functions by applying the Cauchy activation function to suitable linear projections of the input. This approach not only retains the analytic expressiveness of the Cauchy activation function but also offers scalability and adaptability in learning complex features from high-dimensional data.
%Again, with a high-order integration scheme, the above approximation formula remains the same, albeit with different values of $\lambda_k$, which is trainable in any case. We conclude that the order of approximation is $o(m^{-p})$ for any integer $p$~\cite{XNet1}.
%Cauchy's integral formula guarantees the accuracy of the above approximation if enough points on $P$ are taken. However, there is no reason that the sample points must be taken on any specific $P$ to achieve the best approximation. Indeed, our neural network will learn, from training data, where the best sample points are.
\subsection{Using Cauchy activation to approximate high dimensional functions}
\label{Extend to high dimension}
The Cauchy approximation formula derived above can be computationally inefficient when the dimension $N$ is high, due to the large number of multiplicative terms in the denominator. Therefore, for large $N$, we opt for a simplified representation of the function by applying the Cauchy activation function to linear combinations of the variables. Generally, this corresponds to a dual representation of the function, which is especially efficient for feature-finding in high-dimensional problems. Specifically, we approximate the target function \( f(x_1, x_2, \dots, x_N) \) by
\begin{align}
f(x_1, x_2, \dots, x_N) \approx \sum_{k=1}^m \Phi(W_{k1} x_1 + W_{k2} x_2 + \dots + W_{kN} x_N + b_k; \mu_{k1}, \mu_{k2}, d_k), \label{eq:multi_cauchy_activation}
\end{align}
where each \( \Phi \) is a Cauchy activation function as defined in Equation~\eqref{(12)}. Here, the parameters $W_{k1}, W_{k2}, \ldots, W_{kN}$, $b_k, \mu_{k1}, \mu_{k2}$, and \( d_k \) are trainable, allowing the network to capture the complex relationships among the input variables. The validity of this approximation structure is grounded in the following theorem~\cite{XNet1}:
\begin{theorem}[General Approximation Theorem] \label{thm:general-approximation}
Let \(\Phi\) be a family of functions in \(\mathcal{C}(\mathbb{R}, \mathbb{R})\), the space of continuous functions from $\mathbb{R}$ to $\mathbb{R}$, with the universal approximation property. Define
\[
\Phi^N = \left\{ \phi_a(a_1 x_1 + a_2 x_2 + \cdots + a_N x_N) \mid (a_1, \ldots, a_N) \in \mathbb{R}^N, \phi_a \in \Phi \right\}.
\]
Then, the family \(\Phi^N\) has the universal approximation property in \(\mathcal{C}(\mathbb{R}^N, \mathbb{R})\), i.e., every continuous function in \(\mathbb{R}^N\) can be approximated by a linear combination of functions in \(\Phi^N\), uniformly over compact subsets.
\end{theorem}
Since the Cauchy activation function $\Phi$ possesses the universal approximation property in one dimension (as shown in Equation~\eqref{(12)}), Theorem~\ref{thm:general-approximation} guarantees that the family in $\Phi^N$ \eqref{eq:multi_cauchy_activation} can approximate any continuous function in $\mathbb{R}^N$, uniformly over compact subsets. For completeness, we outline the proofs of this theorem based on Ref.~\cite{XNet1}.
\begin{proof}
The set of all polynomials is dense in $\mathcal{C}(B,\mathbb{R})$ for any compact subset $B\subset \mathbb{R}^N$, by the Stone–Weierstrass Theorem. Since an analytical function $f$ on a compact domain can be approximated arbitrarily well by a polynomial, it suffices to show that each monomial can be approximated, up to an arbitrary order, by the Cauchy activation function.
%we only need to show that any monomials can be approximated, up to an arbitrary order, by Cauchy activation function.
As demonstrated earlier, the activation function $\Phi$ is capable of approximating one-dimensional monomials
%As we have shown, activation function $\Phi$ can approximate one dimensional function
$f(x) = x^k$ for any integer $k$. This immediately implies that $\Phi$, when applied to linear combinations in $\mathbb{R}^N$, can approximate functions of the form
$$
(a_1x_1 + a_2x_2 + \ldots + a_Nx_N)^k, \; \mbox{ for } k\in \mathbb{N},\;
(a_1, \ldots, a_N) \in \mathbb{R}^N.
$$
Fix an integer $k$, the multinomial expansion of the above function yields a linear combination of monomials:
%$C(k+N-1, N-1)$ terms such as
$$x_1^{k_1} x_2^{k_2} \cdots x_N^{k_N}, \; \; k_1 + k_2 + \ldots +k_N
= k.$$
%with the total number of terms given by the combinatorial coefficient $C(k+N-1, N-1)$.
%is combinatorial number $k+N-1$ choosing $N-1$.
Conversely, by the Waring decomposition~\cite{ranestad2000varieties}, any such multivariate monomial can itself be expressed as a linear combination of functions of the form
$$
(a_1x_1 + a_2x_2 + \ldots + a_Nx_N)^k.
$$
Hence, the Cauchy activation function is capable of approximating any polynomial function and therefore any analytic function on compact domains in $\mathbb{R}^N$, establishing its expressive power in high dimensions.
%We prove this by induction on the dimension $N$:
% For $N=1$, this is obviously true.
% For $N=2$,\be (x_1+ 0 x_2)^k &=& x_1^k \nn \\
% (x_1 + 1x_2)^k &=& x_1^k + C(k, 1) x_1^{k-1}x_2 + \ldots + C(k, k-1)
% x_1 x_2^{k-1} + x_2^k \nn \\
% (x_1 + 2x_2)^k &=& x_1^k + C(k, 1) 2x_1^{k-1}x_2 + \ldots + C(k, k-1)
% 2^{k-1}x_1 x_2^{k-1} + 2^kx_2^k \nn \\
% (x_1 + 3x_2)^k &=& x_1^k + C(k, 1) 3x_1^{k-1}x_2 + \ldots + C(k, k-1)
% 3^{k-1}x_1 x_2^{k-1} + 3^kx_2^k \nn \\
% &...& \nn \\
% (x_1 + kx_2)^k &=& x_1^k + C(k, 1) kx_1^{k-1}x_2 + \ldots + C(k, k-1)
% k^{k-1}x_1 x_2^{k-1} + k^kx_2^k \nn \ee This is a system of equations for monomials of the form
% $x_1^j x_2^{k-j}$. The coefficient matrix is obviously
% nonsingular, therefore each $x_1^j x_2^{k-j}$ can be solved as a linear
% combination of $(x_1 + jx_2)^k$, with $j=0, 1, \ldots, k$.
%
% For $N=3$, fix any real number $l$, we can expand $((x_1+ lx_2) + j x_3)^k$ for
% $j =0, 1, \ldots , k$ into the product of the form
% $$(x_1 + l x_2)^{k-k_3} x_3^{k_3}, \; \mbox{ for } k_3=1,...,k-1.$$
% Same as the
% previous case with $N=2$, each of the above terms can be expressed as
% a linear combination of
% $$
% ((x_1+ lx_2) + j x_3)^k, \; \mbox{ for
% }\; j =0, 1, \ldots, k.
% $$
% Now for any fixed $k_3$, we choose $l = 0, 1, \ldots, k -k_3$ and expand
% $$(x_1 + l x_2)^{k-k_3} x_3^{k_3}.$$
% Similar to the case with $N=2$, we now obtain $k-k_3$ independent
% equations. From these equations, we can solve each monomial
% $$x_1^{k_1}x_2^{k_2}x_3^{k_3}, \; \mbox{ for } k_1 + k_2 + k_3 =k$$
% as a linear combination of functions of the form $(x_1 + lx_2 +
% jx_3)^k$, with $j=0, 1, \ldots, k$, $l = 0, 1, \ldots, k -k_3$.
%
% For any $N >3$, we continue this process. Any monomial of the form
% $$x_1^{k_1} x_2^{k_2} \cdots x_N^{k_N}, \; \; k_1 + k_2 + \ldots, k_N
% = k$$
% can be written as a linear
% combination of functions of the form
% $$(a_1x_1 + a_2x_2 + \ldots + a_Nx_N)^k$$
% with $a_1=1$; $a_2 = 0, \ldots, k_1$; $a_3 =0, \ldots, k_1 + k_2$;
% $\ldots$; and $a_n = 0, 1, \ldots, k$.
\end{proof}
Cauchy activation offers a uniquely expressive framework for approximating complicated functions. A single layer with Cauchy activation achieves high precision in practice, often eliminating the need for multiple activation layers. While other architectural components (e.g., LSTMs) may still be necessary for domain-specific tasks~\cite{XNet2}, the simplicity of this design reduces computational overhead while maintaining accuracy.
\section{Numerical Experiments}
\label{Sec4}
CompleX-PINN is uniquely characterized by a single hidden layer, referred to as the \textbf{Cauchy layer}. Unlike other neural networks, each neuron in the Cauchy layer is parameterized by trainable parameters $\{\mu_1, \mu_2, d\}$, which provide greater flexibility in capturing complex solution patterns. By default, the parameters $\{\mu_1, \mu_2, d\}$ are initialized to $0.1$, unless otherwise specified in particular experiments. Denote the number of neurons in the Cauchy layer as $N_{\text{Cauchy}}$, then the total number of parameters for a $D$-dimensional PDE with the 1-dimensional output is \[(D+1+3)\times N_{\text{Cauchy}} + (N_{\text{Cauchy}} + 1)\times1.\]
Although each neuron introduces three additional parameters, the overall number of trainable parameters in compleX-PINN remains significantly smaller than currently used multi-layer architectures, as a single Cauchy layer is sufficient.
CompleX-PINN is evaluated against three state-of-the-art models representing distinct technical approaches:
\begin{itemize}
\item Residual-Based Attention PINN (RBA-PINN)~\cite{RRef4}. This method adaptively adjusts the residual loss weights at individual training points to emphasize more critical regions in the solution domain. It employs a standard fully connected neural network with depth $D_{\text{RBA}}$ and width $N_{\text{RBA}}$. The total number of trainable parameters is given by
\[(D+1) \times N_{\text{RBA}} + \sum_{i=1}^{D_{\text{RBA}} - 1}(N_{\text{RBA}}+1) \times N_{\text{RBA}} + (N_{\text{RBA}} + 1)\times 1.\]
\item Binary structured PINN (BsPINN)~\cite{ref34}. This method employs a hierarchical binary architecture that systematically reduces inter-neuron connections, enhancing computational efficiency while maintaining expressive power. The network configuration is denoted as $M$-\(\tilde{M}\), where the first hidden layer contains $M$ neurons grouped in a single block, and each subsequent layer halves the block size until it reaches \(\tilde{M}\). This structure leads to $D_{\text{BsPINN}}:=\log_2(M/\tilde{M})+1$ hidden layers. The total number of parameters is calculated as
\[(D+1) \times M + \sum_{i=1}^{D_{\text{BsPINN}}-1} (2^{1-i}M+1)\times M + (M+1)\times 1.\]
\item Physics-Informed Kolmogorov-Arnold Network (PIKAN)~\cite{liu2024kan,shukla2024comprehensive,wang2025kolmogorov}. PIKAN\footnote{The code of KAN is adapted from the efficientKAN \href{https://github.com/Blealtan/efficient-kan}{https://github.com/Blealtan/efficient-kan}.} leverages the Kolmogorov-Arnold representation theorem by replacing traditional linear transformations with univariate B-spline functions. Given a spline order $p$ and $G$ intervals (which is equivalent to $G+1$ grid points), each univariate function is represented by $G+p$ basis functions. Each layer includes two additional learnable weight components—both initialized to $1$~\cite{liu2024kan}—corresponding to the base function and the spline function. These components contribute to the overall parameter count. With network width $N_{\text{KAN}}$ and depth $D_{\text{KAN}}$, as indicated in Ref.~\cite{wang2025kolmogorov}, the total number of parameters is
\[D\times N_{\text{KAN}}\times(G + p +2) + \sum_{i=1}^{D_{\text{KAN}} - 1} N_{\text{KAN}}\times N_{\text{KAN}}\times(G + p +2) + N_{\text{KAN}}\times 1\times(G + p+2).\]
\end{itemize}
The experimental setup is organized into distinct subsections, each targeting specific objectives, as outlined below:
\begin{itemize}
\item \textbf{Section~\ref{Sec Helm}}:
This section focuses on 2D Helmholtz problems, for which all three models have published results (albeit with varying PDE parameters). To ensure a fair and consistent comparison, we directly adopt the reported results from the original references, using identical training settings for each model. This avoids potential discrepancies due to reimplementation and preserves alignment with established baselines.
\item \textbf{Section~\ref{Sec 3D Helmholtz}-\ref{Section Poisson} (high-dimensional PDEs)}: In these sections, we explore high-dimensional PDEs by reproducing state-of-the-art methods using their publicly available codebases, adapted to our specific target equations. To ensure fair comparisons, we adhere to a standardized evaluation protocol across all methods. Moreover, to mitigate biases due to hyperparameter sensitivity and randomness in training, we perform extensive hyperparameter tuning for each model:
% We investigate high-dimensional PDEs by reproducing state-of-the-art models using their publicly available codes adapted to our target equations, ensuring fair comparisons via standardized protocols. Moreover, to mitigate biases from hyperparameter sensitivity and stochasticity, we conduct extensive tuning of architecture-specific hyperparameters for baseline models.{\color{blue}
\begin{itemize}
\item \textbf{RBA-PINN}: depth $D_{\text{RBA}}\in\{3, 4, 5\}$; width $N_{\text{RBA}}\in\{50, 80, 120, 200\}$.
\item \textbf{BsPINN}: block size configurations: $\{128-8, 128-16, 256-16, 256-32, 512-32, 512-64\}$.
\item \textbf{PIKAN}: number of intervals $G \in \{2, 4, 9\}$; width $N_{\text{KAN}} \in \{5, 10, 15\}$, with depth fixed as $D_{\text{KAN}} = 2$ and spline order $ p =3$, in accordance with common configurations used in prior works~\cite{shukla2024comprehensive,wang2025kolmogorov}
\end{itemize}
Following the tuning process, we report the experimental results corresponding to the best-performing configurations, selected based on validation performance.
% {\color{blue}Following this tuning, only the optimal configurations—selected based on validation performance—are reported in the paper alongside their corresponding results.}
\end{itemize}
%{\color{blue}{\bf Counting the number of parameters.} We will briefly outline how do we count the number of parameters for each model when working on a $d$-dimensional PDE and with the 1-dimensional output.
%\begin{itemize}
% \item \textbf{compleX-PINN}: The Cauchy layer has width $N_{\text{Cauchy}}$. Its total number of parameters is $(d+3)N_{\text{Cauchy}} + N_{\text{Cauchy}} + 1$.
% \item \textbf{RBA-PINN}: The parameter count is given by
% $\left(d \times N_{\text{RBA}} + N_{\text{RBA}} \times N_{\text{RBA}} \times (D_{\text{RBA}} - 1) + N_{\text{RBA}} \times 1\right) + \left(N_{\text{RBA}} \times D_{\text{RBA}} + 1\right)$.
% \item \textbf{BsPINN}: For an $M$-$\tilde{M}$ architecture with $D_{\text{BsPINN}} = \log_2(M/\tilde{M}) + 1$ hidden layers, The total number of biases is $M \times D_{\text{BsPINN}} + 1$ and total number of weights is $d \times M + M^2 \left(\sum_{i=1}^{D_{\text{BsPINN}}-1} 2^{-i}\right) + M \times 1$.
% \item \textbf{PIKAN}: For a 2-layer architecture with spline order $p = 3$:
% $N_{\text{KAN}} \times (d + N_{\text{KAN}} + 1) \times (K + p + 2)$ parameters.For detailed derivations, see \cite{liu2024kan}.
%\end{itemize}}
{\bf Training and testing points.} For 2D spatial or 1D spatio-temporal PDEs, test points are generated on a uniform $300\times 300$ grids (90,000 points). For higher-dimensional problems, 90,000 test points are randomly sampled to balance computational and memory constraints. Training points are also randomly selected, with $N_f$, $N_b$, and $N_0$ denoting the numbers of residual, boundary, and initial condition points, respectively. Note that training points are not necessarily included in the testing set.
%For 2D spatial or 1D spatio-temporal PDEs, test points are generated on uniform $300\times 300$ grids (90,000 points), while higher-dimensional problems use 90,000 randomly sampled points to balance computational and memory constraints. Training points are also randomly selected, with counts denoted as $N_f$ for the residual, $N_b$ for the boundary conditions, and $N_0$ for the initial conditions. Note that training points are not necessarily to be included in the testing set.
%The experimental framework proceeds as follows: First, Section~\ref{Sec Helm} and Section~\ref{Sec Mix-flowing} test compleX-PINN using state-of-the-art PINN training techniques~\cite{RRef4,cho2023separable}. Section~\ref{Sec Klein-Gordon} compares its performance with the Physics-Informed Kolmogorov-Arnold Network (PIKAN)~\cite{liu2024kan,wang2025kolmogorov,shukla2024comprehensive,jacob2024spikans}. While the Cauchy activation function is derived for 1D problems (Section~\ref{Sec 3.1}), it naturally extends to high dimensions (Section~\ref{Sec 3.2}), offering distinct advantages for high-dimensional PDEs. To demonstrate this, we conclude with experiments on 5D and 20D Poisson equations in Section~\ref{Section 5D Poisson}.
%For low-dimensional cases, test points are generated on uniform grids: $300\times 300$ grids (90,000 points) for 2D spatial or 1D spatio-temporal PDEs. Higher-dimensional problems use 90,000 randomly sampled points to balance computational memory constraints. Training points are also randomly selected, with counts denoted as $N_f$ for the residual, $N_b$ for the boundary conditions, and $N_0$ for the initial conditions. Note that training points are not necessarily to be included in the testing set.
%In Sections~\ref{Sec Helm},~\ref{Sec Mix-flowing}, and~\ref{Sec Klein-Gordon}, compleX-PINN is compared against results from Refs.~\cite{RRef4,cho2023separable,jacob2024spikans} with the same training settings, which serve as benchmarks for these canonical problems due to their identical experimental setups. We leverage reported results directly, avoiding re-implementation to ensure consistency with established baselines.
%For high-dimensional problems in Section~\ref{Section 5D Poisson}, comparisons include the state-of-the-art Binary-Structured PINN (BsPINN)~\cite{ref34} and PIKAN, highlighting compleX-PINN’s advantages in high-dimensional spatial domains. Reproductions use publicly available codes adapted to our framework. Fairness is ensured via two metrics: GPU training time per 100 iterations and parameter count. Computational resources are standardized through identical training durations, and parameter counts are reported to quantify efficiency differences. Results are presented for both models trained to match compleX-PINN’s total iterations and those constrained by equivalent GPU training time, enabling rigorous, multi-faceted comparisons.
%To make our best efforts to minimize biases from hyperparameter sensitivity and stochasticity, extensive tuning for the architecture-specific hyperparameters is conducted for baseline models. For PIKAN, we test grid resolutions $G\in\{3,5,10\}$ to assess sensitivity to spatial discretization. For PirateNet, we vary the number of residual blocks $R\in\{2, 3, 4\}$ to explore the impact of network complexity. For BsPINN, we test different block size configurations $\{128-8, 256-16, 512-32\}$, where $M-\tilde{M}$ means the first hidden layer has a block size of $M$ and each subsequent layer halves the block size until reaching $\tilde{M}$.
For each PDE, each model is trained and evaluated across three independent trials using different random seeds. The best-performing result among the trials is reported. Performance is measured using the relative $L^2$ error and the $L^{\infty}$ norm, defined as follows:
\begin{align}
\text{(Relative) } L^2 \text{ error} & = \frac{\sqrt{\sum_{k=1}^N|\hat{u}(\mathbf{x}_k, t_k) - u(\mathbf{x}_k, t_k)|^2}}{\sqrt{\sum_{k=1}^N |u(\mathbf{x}_k, t_k)|^2}},
\label{L2error}\\
L^{\infty} \text{ norm} & = \max_{1 \leq k \leq N} \left|\hat{u}(\mathbf{x}_k, t_k) - u(\mathbf{x}_k, t_k)\right|,\label{Linferror}
\end{align}
where $u$ denotes the ground truth solution, $\hat{u}$ is the prediction from the tested model, and $N$ is the number of testing points.
\input{experiment-revised}
\section{Conclusion and Future Work}
\label{Sec discussion}
We propose compleX-PINN, a novel single-layer physics-informed neural network architecture grounded in the Cauchy integral formula.
Theoretically, this framework reduces parameter complexity while maintaining rigorous approximation guarantees, directly addressing the inefficiencies and depth-related limitations of conventional PINNs.
Empirically, compleX-PINN achieves state-of-the-art accuracy and stability across a variety of high-dimensional PDE benchmarks, consistently delivering orders-of-magnitude lower relative $L^2$ and $L^\infty$ errors than leading alternatives such as RBA-PINN, BsPINN, and PIKAN.
By uniting mathematical rigor with practical scalability, compleX-PINN offers a lightweight yet powerful paradigm for physics-informed machine learning.
%Notably, it exhibits minimal error degradation under dimensionality scaling—a critical advantage over baseline models, which suffer from pronounced instability as problem complexity grows. Computationally, compleX-PINN strikes a scalable balance between efficiency and precision: its per-iteration overhead grows marginally compared to alternatives, yet rapid convergence ensures superior accuracy even under constrained training budgets. By integrating mathematical rigor with practical scalability, our work addresses long-standing challenges in modeling high-dimensional systems, establishing a lightweight yet powerful paradigm for physics-informed machine learning.
%{\color{blue}These advancements stem from the synergy between the Cauchy integral’s analytic structure and neural network design. Unlike traditional PINNs, which rely on deep architectures to compensate for spectral bias, compleX-PINN inherently encodes smoothness through its activation function. This eliminates the need for complex parameterizations while stabilizing gradient dynamics—key to its success in high-dimensional regimes.}
Looking ahead, a key direction is the extension of compleX-PINN to non-smooth and discontinuous systems, such as PDEs with shock waves or sharp interfaces. While the current framework excels in smooth or analytic settings, its reliance on the analyticity assumptions inherent in the Cauchy integral formula may limit its direct applicability to such problems. This limitation indicates a potential future research direction. We will conduct deeper research in this area to explore effective solutions. Furthermore, deep or hierarchical variants of compleX-PINN—such as networks composed of multiple Cauchy layers—could enhance its expressiveness for multiscale phenomena without compromising training efficiency. Finally, integrating compleX-PINN with operator learning frameworks presents an exciting opportunity for emulating parametric systems efficiently, bridging the gap between classical numerical methods and modern data-driven approaches.
%Looking forward, extending compleX-PINN to non-smooth systems—such as PDEs with discontinuous solutions or shock waves—presents a compelling direction. While the current framework excels for analytic or smooth functions, its reliance on the analyticity of the Cauchy integral formula might poses challenges for non-smooth or discontinuous systems. For instance, solutions with sharp gradients or shocks may require hybrid architectures combining Cauchy activations with adaptive sampling or entropy regularization. Future work will explore these adaptations to broaden the framework’s applicability. Similarly, exploring deeper variants of compleX-PINN (e.g., hierarchical compositions of Cauchy layers) may enhance its capacity for multiscale phenomena without sacrificing efficiency. Beyond solving PDEs, integrating the framework with operator learning could unlock fast emulation of parametric systems, bridging the gap between numerical analysis and data-driven modeling.
% \section{Conclusion}
% We propose compleX-PINN, a novel single-layer physics-informed neural network grounded in the Cauchy integral formula. Theoretically, this framework reduces parameter complexity while preserving rigorous approximation guarantees, addressing the inherent inefficiencies of traditional PINN architectures. Empirically, compleX-PINN demonstrates state-of-the-art accuracy and stability across high-dimensional problems, achieving orders-of-magnitude lower relative $L^2$ norms and $L^{\infty}$ norms, compared to existing methods like RBA-PINN, BsPINN, and PIKAN. Notably, it exhibits minimal error degradation under dimensionality scaling—a critical advantage over baseline models, which suffer from pronounced instability as problem complexity grows.
% Computationally, compleX-PINN strikes a scalable balance between efficiency and precision: its per-iteration overhead grows marginally compared to alternatives, yet its rapid convergence ensures superior accuracy even under constrained training budgets. The combination of its lightweight architecture, strong theoretical basis, and empirical effectiveness establishes compleX-PINN as a groundbreaking framework for solving high-dimensional PDEs. It efficiently addresses persistent challenges in balancing accuracy and stability, modeling nonlinear interactions, and enforcing boundary conditions. By integrating mathematical rigor with practical scalability, our work makes meaningful progress in physics-informed machine learning, addressing long-standing challenges in modeling high-dimensional systems with both theoretical soundness and computational efficiency.
%In this paper, we introduce compleX-PINN, which is a single-layer network that utilizes the Cauchy integral formula. Complementing our theoretical advances, the empirical results show that compleX-PINN can achieve a much lower relative $L^2$ error with fewer iterations. Also, the number of parameters in compleX-PINN is much smaller than in the traditional PINN. This enhanced efficiency and accuracy validate the efficacy of our theoretical contributions and highlight the practical advantages of compleX-PINN.
\section*{Acknowledgments}
This work was partially supported by the National Natural Science Foundation of China (72495131, 82441027), Guangdong Provincial Key Laboratory of Mathematical Foundations for Artificial Intelligence (2023B1212010001), Shenzhen Stability Science Program, and the Shenzhen Science and Technology Program under grant no. ZDSYS20211021111415025.
\bibliographystyle{unsrt}
\bibliography{ref}
\end{document}
\subsection{2D Helmholtz equation}
\label{Sec Helm}
%To demonstrate the advantages of compleX-PINN,
The Helmholtz equation, widely used to model wave propagation and diffusion phenomena, describes physical processes that evolve over spatial or spatio-temporal domains. The two-dimensional (2D) Helmholtz equation has been extensively studied in the literature \cite{RRef4, ref21, ref22, ref36, son2023enhanced, ref18, shukla2024comprehensive, ref34}. Therefore, we compare compleX-PINN directly with existing results using the same problem setups reported in these studies.
We consider the following form of the 2D Helmholtz equation:
\begin{align}
&u_{xx}+u_{yy} + k^2u - q(x,y) = 0, \quad (x,y) \in \Omega, \\
&u(x,y) = 0, \quad (x,y)\in\partial\Omega,
\end{align}
where the source term $q(x,y)$ is defined as:
\begin{align}
q(x,y) = k^2\sin(a_1\pi x)\sin(a_2\pi y)-(a_1\pi)^2\sin(a_1\pi x)\sin(a_2\pi y)-(a_2\pi)^2\sin(a_1\pi x)\sin(a_2\pi y).
\end{align}
We set $k = 1$ and $\Omega = [-1, 1]\times [-1,1]$. The exact solution to the equation is given by
\begin{align}
u(x,y) = \sin(a_1\pi x)\sin(a_2\pi y),
\end{align}
where the parameters $a_1$ and $a_2$ control the spatial frequency components of the solution.
\subsubsection{Comparison with baseline models}
We evaluate compleX-PINN against several baseline models: RBA-PINN~\cite{RRef4}, PIKAN~\cite{shukla2024comprehensive,wang2025kolmogorov}(with $a_1 = 1$ and $a_2 = 4$), and BsPINN~\cite{ref34} (with $a_1 = a_2 = 8$). Each model is tested under its original PDE configuration, and we strictly follow the training protocols specified in the respective studies. This approach ensures fair, architecture-focused comparisons by eliminating confounding factors such as differences in hyperparameter tuning.
\paragraph{RBA-PINN Comparison (Case 4.1.1a):}\label{sec:411a} We construct a compleX-PINN model with 500 neurons in the Cauchy layer. For comparison, the RBA-PINN employs a 5-layer fully connected neural network with 128 neurons per layer. To enforce boundary conditions, we use a hard constraint formulation:
\[\hat{u} = (1-x^2)(1-y^2)\hat{u}_{\mathcal{NN}}\]
where $\hat{u}_{\mathcal{NN}}$ is the raw network output and $\hat{u}$ is the final prediction satisfying the PDE boundary conditions.
To ensure comparability, we adopt the same number of training points $N_f = 256,000$ and the same number of iterations as in RBA-PINN~\cite{RRef4}. Specifically, we train using the Adam optimizer for 20,000 iterations with a learning rate of 0.005, followed by 1,000 iterations of L-BFGS for fine-tuning. An exponential learning rate scheduler with decay rate 0.7 is applied every 1,000 steps. All training details—including learning rates, scheduling, and network structure—closely follow the publicly available code provided by RBA-PINN.
%To ensure comparability with prior work, we adopt the same number of training points $N_f$ and training iterations as those reported in RBA-PINN~\cite{RRef4}. In detailed, we train the model by Adam optimizer for 20k iterations and switch to L-BFGS optimizer for 1k iterations to do fine-tuning. The learning rate for Adam is set to be $0.005$. We also employ an exponential scheduler with weight decay rate of $0.7$ for every 1k step. All the implementation details such as learning rates, scheduler settings, and network structure, strictly follow the protocols outlined in the publicly available code provided by RBA-PINN.
%Given that RBA-PINN has already demonstrated superior performance to other state-of-the-art models for this benchmark, we restrict our comparison to its reported results. We plot the relative $L^2$ error history for the Adam training in Fig.~\ref{Helmholtz L2 HIST}. The blue curves represent RBA-PINN, reproduced using the authors’ official code.
%is the history of RBA model, that we directly run the code that the author provides.
Given that RBA-PINN has demonstrated state-of-the-art performance on this benchmark, we compare directly with its reported results. The relative $L^2$ error history during Adam training is shown in Figure~\ref{Helmholtz L2 HIST}, where the blue curve represents the RBA-PINN output using its official code. After the Adam phase, compleX-PINN achieves a relative $L^2$ error of $2.39 \times 10^{-5}$, significantly outperforming the RBA-PINN’s corresponding error of $1.18 \times 10^{-4}$. After the L-BFGS fine-tuning phase, compleX-PINN further reduces the error to $7.51 \times 10^{-7}$, nearly one order of magnitude lower than RBA-PINN’s reported post-L-BFGS error of $5.18 \times 10^{-6}$. The final prediction results are visualized in Figure~\ref{2D Helmholtz Prediction}.
\begin{figure}[!htb]
\centering
\begin{minipage}{0.6\textwidth}
\centering
\includegraphics[width=\linewidth]{Helm-XNet-L2-HIST.png}
\end{minipage}
\caption{Relative $L^2$ error history of the 2D Helmholtz equation using the Adam optimizer (red: compleX-PINN; blue: RBA-PINN), followed by 1,000 iterations of the L-BFGS optimizer. A dashed vertical line indicates the transition from Adam to L-BFGS. The Adam optimizer is configured with a learning rate of 0.005 and an exponential decay scheduler (decay rate: 0.7; decay step: 1,000). Although the large initial learning rate causes some oscillation in the first 5,000 iterations, the relative $L^2$ error stabilizes as the learning rate decays. The subsequent L-BFGS phase further refines the solution.}\label{Helmholtz L2 HIST}
\end{figure}
\begin{figure}[!htb]
\centering
\begin{minipage}{0.45\textwidth}
\centering
\includegraphics[width=\linewidth]{Helm-XNet-Pred.png}
\end{minipage}
\begin{minipage}{0.45\textwidth}
\centering
\includegraphics[width=\linewidth]{Helm-XNet-Error.png}
\end{minipage}
\caption{Prediction and absolute point-wise errors of compleX-PINN for the 2D Helmholtz equation. The results are obtained after fine-tuning with the L-BFGS optimizer. The final relative $L^2$ error achieved is $7.51 \times 10^{-7}$.}\label{2D Helmholtz Prediction}
\end{figure}
\paragraph{PIKAN Comparison (Case 4.1.1b):}\label{sec:411b}
To further validate the performance of compleX-PINN, we compare it with PIKAN~\cite{wang2025kolmogorov} using the same training protocol reported in~\cite{wang2025kolmogorov}. The PIKAN model consists of a single hidden layer with 10 neurons, the number of intervals $G = 5$, and spline order of $p = 3$. The architecture uses a multi-grid training strategy where the number of grid intervals doubles every 600 iterations. This results in a total of 750 parameters as reported in Ref.~\cite{shukla2024comprehensive}. %That experiment was designed to compare PIKAN with an MLP having a similar parameter count.
Accordingly, we configure compleX-PINN with 110 neurons in the Cauchy layer, resulting in a total of 771 parameters, comparable to PIKAN.
We use the same number of training points ($N_f = 2401$, $N_b = 200$), the same loss weights ($\lambda_F = 0.01$, $\lambda_B = 1$), and adopt the L-BFGS optimizer as done in Ref.~\cite{shukla2024comprehensive}. All models are trained for exactly 1,800 iterations. With identical training data and hyperparameters, compleX-PINN achieves a relative $L^2$ error of $6.58 \times 10^{-2}$, substantially outperforming the reported PIKAN result of $4.76 \times 10^{-1}$ by nearly an order of magnitude.
%Having validated compleX-PINN’s superiority under RBA-PINN’s setup, we now evaluate it against PIKAN~\cite{wang2025kolmogorov} using their original training protocol to ensure rigorous comparability. In detail, the PIKAN architecture utilizes a single layer with 10 neurons, with grid number $G = 4$ and spline order $p = 3$. Moreover, PIKAN employs a multi-grid approach: After every 600 training iterations on a given grid, the number of grid points is doubled. Therefore, the PIKAN has 690 parameters in total as reported in Ref.~\cite{shukla2024comprehensive}. The experiment in Ref.~\cite{shukla2024comprehensive} aims to compare PIKAN with MLP with a similar number of parameters. Therefore, we set the width in the Cauchy layer of compleX-PINN to be 110, which only contains 661 parameters.
%To maintain fairness while adapting to our framework, compleX-PINN employs the same number of training points $N_f = 2401$ and $N_b = 200$ (50 per boundary), loss weights $\lambda_f = 0.01$ and $\lambda_B = 1$, and L-BFGS optimizer as specified in Ref.~\cite{shukla2024comprehensive}. Training for exactly 1,800 iterations as in the baseline, compleX-PINN achieves a relative $L^2$ error of $6.58\times 10^{-2}$, surpassing KAN’s reported $4.76\times 10^{-1}$ by almost one order of magnitude. \my{You need to mention the difference in the architecture. The training point and the hyperparameters are the same.}
%employ the Adam optimizer with an initial learning rate of $5\times 10^{-4}$ and an exponential learning rate scheduler (decay rate = 0.9), which halts decay once the learning rate reaches $5\times 10^{-5}$. Training proceeds for 50k iterations without L-BFGS fine-tuning, ensuring consistency in optimization strategy while isolating the impact of reduced training points and adaptive learning rates on performance. Under these conditions, compleX-PINN achieves a relative $L^2$ error of $8.43\times 10^{-5}$, surpassing KAN’s reported $1.16\times 10^{-3}$ by more than one order of magnitude. \my{You need to mention the difference in the architecture. Training point and the hyperparameters are same.}
\paragraph{BsPINN Comparison (Case 4.1.1c):}\label{sec:411c}
We now evaluate compleX-PINN against BsPINN on a more challenging 2D Helmholtz problem with parameters $a_1 = a_2 = 8$, consistent with the setup in Ref.~\cite{ref34}. The BsPINN model uses a 256-16 binary configuration with five hidden layers organized hierarchically. Specifically, the first hidden layer contains one block of 256 neurons; the second layer has two blocks of 128 neurons each; the third has four blocks of 64 neurons; the fourth has eight blocks of 32 neurons; and the fifth has sixteen blocks of 16 neurons. This architecture results in 124,929 total parameters. In contrast, our compleX-PINN model reuses the architecture from Case~\ref{sec:411a} (500 neurons in the Cauchy layer), which has only 3,501 parameters—approximately 3\% of BsPINN’s size—highlighting the efficiency of our design.
%We now consider the Helmholtz equation with higher parameter values $a_1 = a_2 = 8$, a similar problem adopted in BsPINN’s prior work~\cite{ref34}, and replicate its training setup identically. The BsPINN employs a 256-16 configuration with five hidden layers structured recursively: {\color{blue}the first layer contains 1 block of 256 neurons, the second layer contains 2 blocks (128 neurons each), and each subsequent layer $l$ follows the binary halving pattern with \(2^{l-1}\) blocks, each containing \(256/2^{l-1}\) neurons, terminating at 16 neurons in the fifth hidden layer, resulting in 124,929 parameters. For the compleX-PINN, we use the same architecture defined in \textbf{Case \ref{sec:411a}a}, which only contains 3,501 parameters, approximately 3\% of parameters in BsPINN. This significant parameter reduction highlights its advantages in lightweight design and memory efficiency.} \my{Explain the network structure in detail, how many layers? how many parameters?} Specifically, we set $N_f = 6561$ and $N_b= 320$ (80 per boundary), $\lambda_F = 1$ and $\lambda_B = 100$, train with the Adam optimizer for 40k iterations, with the learning rate of $1\times 10^{-3}$.
For both models, we use $N_f = 6561$ interior training points, $N_b = 320$ boundary points, and set loss weights as $\lambda_F = 1$, $\lambda_B = 100$. The models are trained using the Adam optimizer for 40,000 iterations with a learning rate of $1 \times 10^{-3}$.
Under this setup, compleX-PINN achieves a relative $L^2$ error of $5.35 \times 10^{-5}$, significantly outperforming BsPINN’s best reported result of $1.07 \times 10^{-3}$. The relative error history and heatmap of pointwise errors for compleX-PINN are shown in Figures~\ref{Helmholtz L2 HIST-8} and~\ref{2D Helmholtz-8 Prediction}, respectively.
%We results in a relative $L^2$ error of $5.35\times 10^{-5}$, which outperforms the best results of BsPINN ($1.07\times 10^{-3}$ as reported in Ref.~\cite{ref34}) by more than one order of magnitude. {\color{blue}Here, we show the error history and the heatmap of pointwise errors for compleX-PINN in Fig.~\ref{Helmholtz L2 HIST-8} and Fig.~\ref{2D Helmholtz-8 Prediction}, respectively.}\my{you need to mention it is for compleX-PINN only. also, you can not use upload.}
\begin{figure}[!htb]
\centering
\begin{minipage}{0.6\textwidth}
\centering
\includegraphics[width=\linewidth]{Helm-XNet-HIST-8.png}
\end{minipage}
\caption{Relative $L^2$ error and $L^{\infty}$ norm history of compleX-PINN for the 2D Helmholtz equation ($a_1 = a_2 = 8$).}\label{Helmholtz L2 HIST-8}
\end{figure}
\begin{figure}[!htb]
\centering
\begin{minipage}{0.45\textwidth}
\centering
\includegraphics[width=\linewidth]{Helm-XNet-Pred-8.png}
\end{minipage}
\begin{minipage}{0.45\textwidth}
\centering
\includegraphics[width=\linewidth]{Helm-XNet-Error-8.png}
\end{minipage}
\caption{Prediction and absolute point-wise errors of compleX-PINN for the 2D Helmholtz equation when $a_1 = a_2 = 8$. The final relative $L^2$ error after training is $5.35\times 10^{-5}$.}\label{2D Helmholtz-8 Prediction}
\end{figure}
\subsubsection{Robustness of the trainable parameters in Cauchy layers}
\label{Sec Robustness}
Recall that the Cauchy activation function introduced in~\eqref{(12)} contains trainable parameters ${\mu_1, \mu_2, d}$, which are manually initialized. By default, we initialize each of these parameters to 0.1. However, it is important to investigate whether this initialization impacts model performance.
%Recall by the definition of the Cauchy activation function introduced in Eq.~\eqref{(12)}, the set of parameters $\{\mu_1, \mu_2, d\}$ are trainable and initialize manually. We default will initialize each parameters to be 0.1. However, we need to see whether the choice of $\{\mu_1, \mu_2, d\}$ will effect the models in a large magnitude.
In this section, we examine the robustness of these parameters by varying their initial values among $\{0.01, 0.5, 1.0\}$. Testing all possible combinations of $\{\mu_1, \mu_2, d\}$ would be computationally intensive. To balance thoroughness with feasibility, we adopt a controlled perturbation strategy: we fix two parameters at their default value (0.1) and vary the third. This allows us to isolate the effect of each individual parameter’s initialization on the model’s performance.
All experiments follow the same training configuration described in Case~\ref{sec:411a}a, and we restrict our analysis to the Adam training phase.
%In this section, we will vary the initialization of the parameters with $\{0.01, 0.5, 1.0\}$. We recognize that testing all combinations of $\{\mu_1, \mu_2, d\}$ is computationally prohibitive. To balance rigor and feasibility, we adopted a controlled perturbation approach: fixing two parameters (at their default value of 0.1) and varying the third. This isolates the sensitivity of each parameter to initialization. All the training setup will be the same as in the Section~\ref{Sec Helm} while we will only focus on Adam training.
\begin{table}[ht]
\centering
\caption{Robustness of model performance with respect to the initialization of Cauchy activation function parameters ${\mu_1, \mu_2, d}$. Each parameter is varied individually while the other two are fixed at 0.1. Reported values are the relative $L^2$ errors after 20,000 Adam training iterations (mean ± standard deviation over 5 trials). "NaN" indicates numerical instability during training.}
\label{tab:parameter_robustness}
\begin{tabular}{lccc}
\toprule
\multirow{2}{*}{Parameter} & \multicolumn{3}{c}{Initialization Value} \\
\cmidrule{2-4}
& 0.01 & 0.5 & 1.0 \\
\midrule
$\mu_1$ (fixed: $\mu_2=0.1, d=0.1$) & $(2.38\pm 0.18)\times 10^{-5}$ & $(2.39\pm 0.47)\times 10^{-5}$ & $(2.58\pm 0.41)\times 10^{-5}$ \\
$\mu_2$ (fixed: $\mu_1=0.1, d=0.1$) & $(2.69\pm 0.22)\times 10^{-5}$ & $(2.99\pm 0.57)\times 10^{-5}$ & $(3.47\pm 0.83)\times 10^{-5}$ \\
$d$ (fixed: $\mu_1=0.1, \mu_2=0.1$) & NaN & $(3.38\pm 0.46)\times 10^{-5}$ & $(7.85\pm 0.16)\times 10^{-5}$ \\
\bottomrule
\end{tabular}
\end{table}
As shown in Table~\ref{tab:parameter_robustness}, variations in the initial values of $\mu_1$ and $\mu_2$ result in relatively consistent model performance, with only minor fluctuations in relative $L^2$ error. This suggests that the model is robust to the initialization of these two parameters. In contrast, the initialization of $d$ has a more pronounced impact. Specifically, setting $d = 0.01$ leads to numerical instability (NaN results), while larger values increase the relative error. This sensitivity is expected, as $d$ controls the "width" of the Cauchy activation function. As a scale parameter, small values of $d$ sharply localize the activation, potentially leading to gradient instability and poor generalization. These findings highlight the importance of avoiding near-zero initializations for $d$ to ensure stable and effective training.
%We can see while varying the initial values of $\mu_1$ and $\mu_2$, the performances are similar. Moreover, Our tests reveal that $d$ is more sensitive to initialization than $\mu_1$ or $\mu_2$, given that $d$ controls the "width" of the activation curve. This aligns with its role as a scale parameter in the Cauchy function, where small perturbations in $d$ can drastically alter the activation’s shape and stability. Also, NaN result at $d = 0.01$ underscores the necessity of avoiding near-zero initializations for $d$, as they violate the Cauchy function’s numerical stability.
\subsection{3D Helmholtz equation}
\label{Sec 3D Helmholtz}
We now extend our investigation to the 3D Helmholtz equation. Specifically, we consider the following problem:
\begin{align*}
&\Delta u + k^2u = q(x,y,z) , \quad (x,y,z) \in \Omega, \\
&u(x,y,z) = 0, \quad (x,y,z)\in\partial\Omega,
\end{align*}
where $k = 1$, and the analytical solution is given by
\begin{align*}
u(x,y,z) = \sin(2\pi x)\sin(2\pi y)\sin(2\pi z),
\end{align*}
with the source term $q(x, y, z)$ inferred accordingly.
All models incorporate a hard constraint for the boundary condition, expressed as $\hat{u} = (1 - x^2)(1 - y^2)(1 - z^2)\hat{u}_{\mathcal{NN}}$, and use $N_f = 10{,}000$ residual training points. Training is performed using the Adam optimizer for 20k iterations with the same learning rate and exponential decay scheduler as described in Case~\ref{sec:411a}a.
The compleX-PINN employs 500 neurons in the Cauchy layer. For baseline models, hyperparameter tuning is conducted as described at the beginning of Section~\ref{Sec4}, and the best-performing configurations are selected accordingly: RBA-PINN uses $D_{\text{RBA}} = 4$ and $N_{\text{RBA}} = 50$; PIKAN uses $G = 4$ and $N_{\text{KAN}} = 5$; BsPINN adopts a 5-layer hierarchical architecture with block sizes halving from 256 to 16, resulting in a 256–16 configuration.
Table~\ref{tab:3D Helmholtz time} summarizes the number of trainable parameters and the GPU time per 100 training iterations for each model. Notably, compleX-PINN and PIKAN have significantly fewer parameters than RBA-PINN and BsPINN, highlighting their advantage in model compactness and memory efficiency. While compleX-PINN has more parameters than PIKAN, it requires substantially less training time per 100 iterations, demonstrating superior computational efficiency.
\begin{table}[!ht]
\centering
\caption{Number of trainable parameters and GPU training time per 100 iterations for different models. This comparison highlights the computational and memory efficiency of compleX-PINN relative to other baseline methods.}
\label{tab:3D Helmholtz time}
\begin{tabular}{lrr}
\toprule
Model & Number of Parameters & GPU Time (s/100 iters) \\
\midrule
compleX-PINN & 4,001 & 5.23 \\
RBA-PINN & 7,901 & 3.05 \\
BsPINN & 125,185 & 4.81 \\
PIKAN & 405 & 11.70 \\
\bottomrule
\end{tabular}
\end{table}
Table~\ref{tab:3D Helmholtz errors} reports the numerical performance under two evaluation criteria: (i) the same number of training iterations (20k), and (ii) equivalent total training time. After 20k iterations, compleX-PINN achieves a relative $L^2$ error of $5.54 \times 10^{-4}$. PIKAN has the second smallest $L^2$ error of $7.27 \times 10^{-4}$ after 20k iterations, despite its longer training time. When training time is equalized, BsPINN and RBA-PINN attain relative $L^2$ errors of $9.94 \times 10^{-4}$ and $1.03 \times 10^{-3}$, respectively. Overall, compleX-PINN consistently outperforms other methods and yields the best performance across both evaluation metrics, even when trained under time constraints.
%For all models, we impose the hard constraint for the boundary condition $\hat{u} = (1-x^2)(1-y^2)(1-z^2)\hat{u}_{\mathcal{NN}}$ and set the number of residual training points to $N_f = 10,000$. All models are trained using the Adam optimizer for 20k iterations, with the same learning rate and scheduler as described in Case~\ref{sec:411a}a. The compleX-PINN model uses 500 neurons in the Cauchy layer. For the other models, hyperparameter tuning was performed as discussed in Section~\ref{Sec4}. Based on the evaluation of parameter spaces in that section, the optimal configurations were selected as follows: for RBA-PINN, $D_{\text{RBA}} = 4$ and $N_{\text{RBA}} = 50$; for PIKAN, $G = 4$ and $N_{\text{KAN}} = 5$; and for BsPINN, a block size configuration of 256-16 (where the first hidden layer has a block size of 256, and each subsequent layer halves the block size until reaching 16, with a total of 5 hidden layers). Under these settings, Table~\ref{tab:3D Helmholtz time} reports the number of parameters and the GPU time per 100 training iterations for each model. {\color{blue}compleX-PINN has far fewer parameters than BsPINN and RBA-PINN, demonstrating advantages in memory efficiency and model simplicity; while it has more parameters than PIKAN, its training time per 100 iterations is significantly shorter, reflecting better computational efficiency.}
%For the compleX-PINN model, the total number of parameters is $7\times 500 + 501 = 4,001$. By comparison, the RBA-PINN model has $(3\times N_{\text{RBA}} + N_{\text{RBA}} \times N_{\text{RBA}} \times (D_{\text{RBA}} - 1) + N_{\text{RBA}} \times 1) + (N_{\text{RBA}} \times D_{\text{RBA}} + 1) = 7,901$ parameters. For BsPINN, the total number of biases is $256 \times 5 + 1 = 1,281$, and the number of weights is $3 \times 256 + (256 \times 256 \times 2^0) + (128 \times 128 \times 2^1) + (64 \times 64 \times 2^2) + (32 \times 32 \times 2^3) + (16 \times 1 \times 2^4) = 123,904$, yielding a total of $125,185$ parameters.
%For PIKAN, parameter counts are as follows: in the first layer, there are base weights $3 \times 5 = 15$, spline scalers $3 \times 5 = 15$, and spline weights $5 \times (3 \times (5 + 3 - 1) + 3) = 120$, for a total of 150 parameters. In the second layer, there are base weights $5 \times 5 = 25$, spline scalers $5 \times 5 = 25$, and spline weights $5 \times (5 \times (5 + 3 - 1) + 5) = 200$, for a total of 250 parameters. Finally, in the last layer, there are 50 parameters. This results in a total of 450 parameters for PIKAN. For more details on parameter calculations, refer to the literature~\cite{liu2024kan}.
\begin{table}[!ht]
\centering
\caption{Relative $L^2$ errors and $L^{\infty}$ norms for each model under two evaluation settings—fixed number of iterations (20k) and equivalent computational time.}
\label{tab:3D Helmholtz errors}
\begin{tabular}{lcccc}
\toprule
\multirow{2}{*}{\makecell{Model\\(total 20k iter.)}} & \multicolumn{2}{c}{Relative $L^2$ Norm} & \multicolumn{2}{c}{$L^\infty$ Norm} \\
\cmidrule(lr){2-3} \cmidrule(lr){4-5}
& Same Iter. & Same Comput. Time & Same Iter. & Same Comput. Time \\
\midrule
compleX-PINN & \multicolumn{2}{c}{\bm{$5.54 \times 10^{-4}$}} & \multicolumn{2}{c}{\bm{$9.27 \times 10^{-4}$}} \\
RBA-PINN & $2.71 \times 10^{-3}$ & $1.03 \times 10^{-3}$ & $3.13 \times 10^{-2}$ & $3.01 \times 10^{-2}$ \\
BsPINN & $1.03 \times 10^{-3}$ & $9.94 \times 10^{-4}$ & $4.25 \times 10^{-3}$ & $4.18 \times 10^{-3}$ \\
PIKAN & $7.27 \times 10^{-4}$ & $3.58 \times 10^{-3}$ & $2.72 \times 10^{-3}$ & $5.07 \times 10^{-3}$ \\
\bottomrule
\end{tabular}
\end{table}
%Moreover, the numerical results are summarized in Table~\ref{tab:3D Helmholtz errors}. As discussed in Section~\ref{Sec4}, we report results from training for 20k iterations under two conditions: matching the number of training epochs and matching the total training time, which stands for the column of Same Iter. and Same Comput. Time.
%After 20k iterations, compleX-PINN attains a relative $L^2$ error of {\color{purple}$5.54 \times 10^{-4}$}. In contrast, PIKAN, despite being slower, reaches a relative $L^2$ error of {\color{purple}$7.27 \times 10^{-4}$} within the same 20k-iteration regime. When considering the same training time, RBA-PINN and BsPINN achieve relative $L^2$ errors of {\color{purple}$1.03 \times 10^{-3}$ and $9.94 \times 10^{-4}$}, respectively. {\color{blue}While BsPINN is competitive in accuracy, its high parameter count may limit scalability.} Also, our model outperforms PIKAN across both evaluation metrics. Moreover, although our model trains more slowly than RBA-PINN and BsPINN, it still achieves superior performance when evaluated under identical training durations.
\subsection{3D heat equation}
\label{Sec 3D Heat}
We now evaluate model performance on the 3D heat equation given by:
\begin{align}
&\Delta u(\mathbf{x},t) = 12\pi^2 u_t , \quad \mathbf{x} \in \Omega:=[0,1]^3, t\in T:=[0,10], \\
&u(\mathbf{x},t) = 0, \quad \mathbf{x}\in\partial\Omega, t\in T, \\
&u(\mathbf{x},0) = \sin(2\pi x_1)\sin(2\pi x_2)\sin(2\pi x_3), \quad \mathbf{x} \in \Omega.
\end{align}
The exact solution is:
\begin{align}
u(\mathbf{x}, t) = e^{-t}\sin(2\pi x_1)\sin(2\pi x_2)\sin(2\pi x_3).
\end{align}
%We choose the spatial domain $\Omega=[0,1]^3$ and the time domain $T=[0,10]$. The initial condition $u_0$ and parameter $c$ can be derived by the ground truth.
To enforce initial and boundary conditions, we apply a hard constraint of the form:
\[\hat{u} = t \prod_{i=1}^3x_i(1-x_i) \cdot \hat{u}_{\mathcal{NN}} + \prod_{i=1}^3\sin(2\pi x_i).\]
We use $N_f = 10,000$ residual training points across space and time.
The compleX-PINN is configured with 300 neurons in its Cauchy layer. For comparison, the baseline models are optimized with the following architectures based on prior tuning: RBA-PINN uses 4 hidden layers with 80 neurons each; BsPINN adopts a 256-16 block-wise configuration; and PIKAN uses $N_{\text{KAN}} = 10$ and $G = 2$.
%The number of residual training points is set to $ N_f=10,000$. We impose the hard constraint $\hat{u} = \hat{u}{\mathcal{NN}}(t \prod_{i=1}^3x_i(1-x_i)) + \prod_{i=1}^3\sin(2\pi x_i)$, thereby eliminating the need for an initial and boundary loss function in the network. The compleX-PINN model is configured with 300 neurons in the Cauchy layer. In our study, we identified the optimal architectures for several models. RBA-PINN achieves its best performance when configured as a 4-layer network, with each hidden layer consisting of 80 neurons. For BsPINN, the 256-16 architecture yields optimal results. Similarly, PIKAN achieves its peak performance with $N_{\text{KAN}} = 10$ and a grid number of $G = 2$.
All models are trained using the Adam optimizer with an initial learning rate of $5 \times 10^{-3}$, decayed exponentially by a factor of 0.85 every 1,000 iterations, until below $1 \times 10^{-5}$. Figure~\ref{3D Heat HIST} presents the training history of the relative $L^2$ error, and Table~\ref{tab:3D Heat errors} summarizes each model's performance under two evaluation protocols: fixed number of training iterations (20,000) and fixed total training time (matched to compleX-PINN).
%All networks are trained using the Adam optimizer with a learning rate of $5\times 10^{-3}$, followed by an exponential scheduler with a decay rate of 0.85 applied every 1,000 iterations. The decay process is stopped once the learning rate reaches $1\times 10^{-5}$. Fig.~\ref{3D Heat HIST} illustrates the history of the relative $L^2$ errors for these models, while Table~\ref{tab:3D Heat errors} summarizes the final results, including the GPU time per 100 iterations and the performance of models tested under the same training time and iteration count as compleX-PINN.
\begin{figure}[!htb]
\centering
\begin{minipage}{0.45\textwidth}
\centering
\includegraphics[width=\linewidth]{Heat-HIST-l2-iter.png}
\end{minipage}
\begin{minipage}{0.45\textwidth}
\centering
\includegraphics[width=\linewidth]{Heat-HIST-l2-time.png}
\end{minipage}
\caption{Relative $L^2$ error history of compleX-PINN (red), RBA-PINN (blue), BsPINN (cyan), and PIKAN (magenta) on the 3D heat equation. The left plot shows error evolution over 20,000 training iterations. The right plot presents the error history when models are trained for the same total GPU time as compleX-PINN.}\label{3D Heat HIST}
\end{figure}
\begin{table}[!ht]
\centering
\caption{3D heat equation: Relative $L^2$ errors and $L^{\infty}$ norms for each model under two evaluation settings—fixed number of iterations (20k) and equivalent computational time.}
\label{tab:3D Heat errors}
\begin{tabular}{lccccc}
\toprule
\multirow{2}{*}{\makecell{Model\\(total 20k iter.)}} & \multicolumn{2}{c}{Relative $L^2$ Norm} & \multicolumn{2}{c}{$L^\infty$ Norm} & \multirow{2}{*}{\makecell{GPU Time\\(per 100 iter.)}} \\
\cmidrule(lr){2-3} \cmidrule(lr){4-5}
& Same Iter. & Same Comput. Time & Same Iter. & Same Comput. Time & \\
\midrule
compleX-PINN & \multicolumn{2}{c}{\bm{$2.73 \times 10^{-4}$}} & \multicolumn{2}{c}{\bm{$7.67 \times 10^{-4}$}} & \text{4.25s} \\
RBA-PINN & $1.16 \times 10^{-3}$ & $1.06 \times 10^{-3}$ & $2.27 \times 10^{-3}$ & $2.23 \times 10^{-3}$ & \text{3.03s} \\
BsPINN & $1.75 \times 10^{-2}$ & $1.77 \times 10^{-2}$ & $3.58 \times 10^{-2}$ & $3.76 \times 10^{-2}$ & \text{5.05s} \\
PIKAN & $1.10 \times 10^{-3}$ & $3.12 \times 10^{-3}$ & $1.39 \times 10^{-3}$ & $4.19 \times 10^{-3}$ & \text{20.8s} \\
\bottomrule
\end{tabular}
\end{table}
Under identical iteration count, compleX-PINN achieves a relative $L^2$ error of $2.73 \times 10^{-4}$, significantly outperforming the other three methods. This advantage persists even under equal runtime. For example, although RBA-PINN completes over 28,000 iterations within the same time (due to its faster per-iteration speed of 3.03s per 100 iterations), its final error remains at $1.06 \times 10^{-3}$, still nearly 4× higher than compleX-PINN. PIKAN, despite achieving reasonable accuracy ($1.10 \times 10^{-3}$) with 20k iterations, suffers from high computational cost (20.8s per 100 iterations), allowing only ~4,100 iterations under equal runtime. Consequently, its error increases to $3.12 \times 10^{-3}$, over 11× higher than compleX-PINN.
In terms of maximum point-wise error, compleX-PINN again demonstrates superiority, achieving the lowest $L^\infty$ norm of $7.67 \times 10^{-4}$, underscoring its effectiveness in reducing localized spatiotemporal errors. These results highlight the strength of the Cauchy-layer-based architecture in enhancing both global and worst-case accuracy.
\subsection{High-dimensional Poisson equation}
\label{Section Poisson}
We evaluate the performance of compleX-PINN on high-dimensional problems using the Poisson equation in five and ten dimensions. The governing equation is defined on the hypercube domain $\Omega:= [-1,1]^d$, with $d \in {5, 10}$:
\begin{align}
-\Delta u &= f \quad\text{in }\Omega,\\
u &= g \quad\text{on }\partial\Omega,
\end{align}
where the exact solution is given by $u(\mathbf{x}) = \sum_{i=1}^d \sin(\pi x_i)$ and the corresponding source term is $f(\mathbf{x}) = \pi^2 \sum_{i=1}^d \sin(\pi x_i)$. This setup provides a controlled and scalable benchmark for assessing the accuracy and efficiency of the models.
For both the 5D and 10D cases, all models are trained under identical settings: $N_f = 10,000$ interior collocation points, $N_b = 500$ boundary points, and 50,000 training iterations using the Adam optimizer with an initial learning rate of $1 \times 10^{-2}$. A learning rate scheduler with decay factor 0.85 and decay step 1,000 is applied, and the loss weights are fixed at $\lambda_F = 1$ and $\lambda_B = 100$. The compleX-PINN employs 200 neurons in the Cauchy layer. Baseline models are configured with their optimal architectures: RBA-PINN uses 3 hidden layers of 200 neurons; PIKAN adopts $N_{\text{KAN}} = 15$ with number of intervals $G = 9$; and BsPINN follows a 128-8 architecture. The results are summarized in Table~\ref{tab:Poisson errors} and visualized in Figure~\ref{fig:Poisson HIST}.
%As in the previous experiments, for $d = 5$, all models share identical training configurations: $N_f = 10,000$ interior collocation points and $N_b = 500$ boundary points, trained using the Adam optimizer with an initial learning rate of $1\times 10^{-2}$ for 50,000 iterations. An exponential learning rate scheduler is applied with a decay rate of 0.85 and a decay step of 1,000 iterations. The loss weights are set to $\lambda_F = 1$ and $\lambda_B = 100$. The compleX-PINN model incorporates a Cauchy layer with 200 neurons, while the baseline models utilize optimal architectures: RBA-PINN employs 3 hidden layers with 200 neurons each; PIKAN is configured with $N_{\text{KAN}} = 15$ and a grid number of $G = 9$; and BsPINN adopts a 128-8 architecture.
%To ensure a fair comparison of the same model across different values of $d$, the ten-dimensional case uses the same network setup. The numerical results are summarized in Table~\ref{tab:Poisson errors} and Fig.~\ref{fig:Poisson HIST}.
%In the 5D case, compleX-PINN achieves a relative $L^2$ error of {\color{purple}$1.66\times 10^{-5}$} and an $L^{\infty}$ norm of {\color{purple}$4.09\times 10^{-3}$}, outperforming RBA-PINN, BsPINN, and PIKAN by one to two orders of magnitude. When extended to 10D, compleX-PINN demonstrates remarkable stability: its relative $L^2$ error increases only marginally to {\color{purple}$3.44 \times 10^{-5}$} (a growth factor of 2.1), while its $L^{\infty}$ norm rises slightly to {\color{purple}$4.54 \times 10^{-3}$} (a 1.1× increase). In contrast, the baseline models experience larger error degradation. For instance, the relative $L^2$ error and $L^{\infty}$ norm of RBA-PINN grow by 2.1× (from {\color{purple}$2.77 \times 10^{-4}$ to $5.84 \times 10^{-4}$}) and 4.1× (from {\color{purple}$1.58 \times 10^{-2}$ to $6.45 \times 10^{-2}$}), respectively. Similarly, BsPINN’s $L^{\infty}$ norm escalates by 5.9× (from {\color{purple}$1.09 \times 10^{-2}$ to $6.46 \times 10^{-2}$}), and PIKAN’s $L^{\infty}$ norm nearly doubles. These results emphasize compleX-PINN’s exceptional ability to mitigate error propagation in high-dimensional spaces.
In the 5D case, compleX-PINN achieves a relative $L^2$ error of $1.66 \times 10^{-5}$ and an $L^{\infty}$ norm of $4.09 \times 10^{-3}$, significantly outperforming all baselines by one to two orders of magnitude. When extended to 10D, compleX-PINN maintains remarkable robustness, with its $L^2$ error increasing only slightly to $3.44 \times 10^{-5}$ (a 2.1× growth) and its $L^{\infty}$ norm rising to $4.54 \times 10^{-3}$ (1.1× growth). In contrast, baseline models experience much greater degradation from 5D to 10D. For instance, RBA-PINN’s $L^{\infty}$ norm increases by 4.1× (from $1.58 \times 10^{-2}$ to $6.45 \times 10^{-2}$), BsPINN’s by 5.9× (from $1.09 \times 10^{-2}$ to $6.46 \times 10^{-2}$), and PIKAN’s nearly doubles (from $1.57 \times 10^{-1}$ to $3.29 \times 10^{-1}$). These results underscore compleX-PINN’s ability to suppress error amplification in high-dimensional settings.
%{\color{blue}In the 5D case, compleX-PINN achieves a relative $L^2$ error of {\color{purple}$1.66\times 10^{-5}$} and an $L^{\infty}$ norm of {\color{purple}$4.09\times 10^{-3}$}, outperforming RBA-PINN, BsPINN, and PIKAN by one to two orders of magnitude. When extended to 10D, compleX-PINN demonstrates remarkable stability: its relative $L^2$ error increases only marginally to {\color{purple}$3.44 \times 10^{-5}$} (a growth factor of 2.1), while its $L^{\infty}$ norm rises slightly to {\color{purple}$4.54 \times 10^{-3}$} (a 1.1× increase). In contrast, while baseline models exhibit comparable $L^2$ error growth (approximately 2×), their $L^{\infty}$ norms degrade dramatically. For instance, RBA-PINN’s $L^{\infty}$ norm grows by 4.1× (from {\color{purple}$1.58 \times 10^{-2}$} to {\color{purple}$6.45 \times 10^{-2}$}), BsPINN’s $L^{\infty}$ norm escalates by 5.9× (from {\color{purple}$1.09 \times 10^{-2}$} to {\color{purple}$6.46 \times 10^{-2}$}), and PIKAN’s $L^{\infty}$ norm nearly doubles (from {\color{purple}$1.57 \times 10^{-1}$} to {\color{purple}$3.29 \times 10^{-1}$}). These results emphasize compleX-PINN’s exceptional ability to mitigate error propagation in high-dimensional spaces.}
In terms of efficiency, compleX-PINN also demonstrates strong scalability. The GPU time per 100 iterations increases by only 33\% from 4.05s (5D) to 5.38s (10D)—notably lower than the growth observed in RBA-PINN (45\%), BsPINN (47\%), and PIKAN (58\%).
%Despite this modest overhead, compleX-PINN continues to deliver superior accuracy when all models are trained for the same total runtime. %For example, in 10D, RBA-PINN and BsPINN achieve relative errors of {\color{purple}$5.29 \times 10^{-4}$} and {\color{purple}$5.30 \times 10^{-4}$}, respectively—more than an order of magnitude worse than compleX-PINN’s {\color{purple}$3.44 \times 10^{-5}$}.
These results demonstrate that compleX-PINN not only achieves high accuracy in high dimensions but also maintains strong computational efficiency and stability, effectively overcoming the key challenges of high-dimensional PDE solvers.
%Complementing its accuracy, compleX-PINN exhibits scalable computational efficiency. Scaling from 5D to 10D increases the GPU time per 100 iterations by only 33\% (from 4.05s to 5.38s), which is significantly lower than the overheads incurred by RBA-PINN (45\%), BsPINN (47\%), and PIKAN (58\%). Despite this modest computational growth, compleX-PINN’s rapid convergence ensures superior accuracy even when baselines are allocated equivalent total training time (Fig.~\ref{fig:Poisson HIST}). For example, in 10D, RBA-PINN and BsPINN under matched training durations exhibit relative errors of {\color{purple}$5.29\times 10^{-4}$ and $5.30\times 10^{-4}$}, respectively—over an order of magnitude higher than compleX-PINN’s {\color{purple}$3.44\times 10^{-5}$}. By maintaining minimal error accumulation alongside a scalable computational profile, compleX-PINN resolves critical trade-offs between accuracy, efficiency, and stability that hinder conventional high-dimensional solvers.
\begin{table}[ht]
\centering
\caption{5D and 10D Poisson equations: Relative $L^2$ errors and $L^{\infty}$ norms for each model under two evaluation settings—fixed number of iterations (20k) and equivalent computational time.}
\label{tab:Poisson errors}
% 5D Block
\begin{tabular}{lccccc}
\toprule
\multirow{2}{*}{\makecell{Model (5D)\\(total 50k iter.)}} & \multicolumn{2}{c}{Relative $L^2$ Norm} & \multicolumn{2}{c}{$L^\infty$ Norm} & \multirow{2}{*}{\makecell{GPU Time\\(per 100 iter)}} \\
\cmidrule(lr){2-3} \cmidrule(lr){4-5}
& Same Iter. & Same Comput. Time & Same Iter. & Same Comput. Time & \\
\midrule
compleX-PINN & \multicolumn{2}{c}{\bm{$1.66 \times 10^{-5}$}} & \multicolumn{2}{c}{\bm{$4.09 \times 10^{-3}$}} & \text{4.05s} \\
RBA-PINN & $2.77 \times 10^{-4}$ & $2.42 \times 10^{-4}$ & $1.58 \times 10^{-2}$ & $1.40\times 10^{-2}$ & \text{2.86s} \\
BsPINN & $2.19 \times 10^{-4}$ & $2.49\times 10^{-4}$ & $1.09 \times 10^{-2}$ & $1.14 \times 10^{-2}$ & \text{5.11s} \\
PIKAN & $2.64\times 10^{-3}$ & $7.37\times 10^{-3}$ & $1.57\times 10^{-1}$ & $4.18\times 10^{-1}$& \text{21.2s} \\
\bottomrule
\end{tabular}
\vspace{0.5em} % Space between blocks
% 10D Block
\begin{tabular}{lccccc}
\toprule
\multirow{2}{*}{\makecell{Model (10D)\\(total 50k iter.)}} & \multicolumn{2}{c}{Relative $L^2$ Norm} & \multicolumn{2}{c}{$L^\infty$ Norm} & \multirow{2}{*}{\makecell{GPU Time\\(per 100 iter)}} \\
\cmidrule(lr){2-3} \cmidrule(lr){4-5}
& Same Iter. & Same Comput. Time & Same Iter. & Same Comput. Time & \\
\midrule
compleX-PINN & \multicolumn{2}{c}{\bm{$3.44\times 10^{-5}$}} & \multicolumn{2}{c}{\bm{$4.54\times 10^{-3}$}} & \text{5.38s} \\
RBA-PINN & $5.84\times 10^{-4}$ & $5.29\times 10^{-4}$ & $6.45\times 10^{-2}$ & $6.83\times 10^{-2}$ & \text{4.16s} \\
BsPINN & $4.19\times 10^{-4}$ & $5.30\times 10^{-4}$ & $6.46\times 10^{-2}$ & $6.93\times 10^{-2}$& \text{7.52s} \\
PIKAN & $5.48\times 10^{-3}$ & $1.67\times 10^{-2}$ & $3.29\times 10^{-1}$ & $8.48\times 10^{-1}$ &\text{33.6s} \\
\bottomrule
\end{tabular}
\end{table}
\begin{figure}[!htb]
\centering
\begin{minipage}{0.45\textwidth}
\centering
\includegraphics[width=\linewidth]{5D-Poisson-HIST-L2-iter.png}
\end{minipage}
\begin{minipage}{0.45\textwidth}
\centering
\includegraphics[width=\linewidth]{5D-Poisson-HIST-L2-time.png}
\end{minipage}
\begin{minipage}{0.45\textwidth}
\centering
\includegraphics[width=\linewidth]{10D-Poisson-HIST-iter.png}
\end{minipage}
\begin{minipage}{0.45\textwidth}
\centering
\includegraphics[width=\linewidth]{10D-Poisson-HIST-time.png}
\end{minipage}
\caption{Relative $L^2$ error history of compleX-PINN (red), RBA-PINN (blue), BsPINN (cyan), and PIKAN (magenta) for the Poisson equation in 5D (top row) and 10D (bottom row). The left column presents error trajectories over 50,000 training iterations, while the right column compares performance under equal GPU time budgets. compleX-PINN consistently demonstrates faster convergence and superior accuracy across both dimensional settings.}\label{fig:Poisson HIST}
\end{figure}
\begin{thebibliography}{10}
\bibitem{ref1}
M.~Raissi, P.~Perdikaris, and G.~E. Karniadakis.
\newblock Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations.
\newblock {\em Journal of Computational Physics}, 378:686--707, 2019.
\bibitem{ref2}
George~Em Karniadakis, Ioannis~G Kevrekidis, Lu~Lu, Paris Perdikaris, Sifan Wang, and Liu Yang.
\newblock Physics-informed machine learning.
\newblock {\em Nature Reviews Physics}, 3(6):422--440, 2021.
\bibitem{ref3}
Yuyao Chen, Lu~Lu, George~Em Karniadakis, and Luca Dal~Negro.
\newblock Physics-informed neural networks for inverse problems in nano-optics and metamaterials.
\newblock {\em Optics express}, 28(8):11618--11633, 2020.
\bibitem{ref4}
Minglang Yin, Xiaoning Zheng, Jay~D Humphrey, and George~Em Karniadakis.
\newblock Non-invasive inference of thrombus material properties with physics-informed neural networks.
\newblock {\em Computer Methods in Applied Mechanics and Engineering}, 375:113603, 2021.
\bibitem{ref5}
Shengze Cai, Zhicheng Wang, Sifan Wang, Paris Perdikaris, and George~Em Karniadakis.
\newblock Physics-informed neural networks for heat transfer problems.
\newblock {\em Journal of Heat Transfer}, 143(6):060801, 2021.
\bibitem{ref6}
Shengze Cai, Zhicheng Wang, Chryssostomos Chryssostomidis, and George~Em Karniadakis.
\newblock Heat transfer prediction with unknown thermal boundary conditions using physics-informed neural networks.
\newblock In {\em Fluids Engineering Division Summer Meeting}, volume 83730, page V003T05A054. American Society of Mechanical Engineers, 2020.
\bibitem{ref7}
Jiang-Zhou Peng, Yue Hua, Yu-Bai Li, Zhi-Hua Chen, Wei-Tao Wu, and Nadine Aubry.
\newblock Physics-informed graph convolutional neural network for modeling fluid flow and heat convection.
\newblock {\em Physics of Fluids}, 35(8), 2023.
\bibitem{ref8}
Jinshuai Bai, Timon Rabczuk, Ashish Gupta, Laith Alzubaidi, and Yuantong Gu.
\newblock A physics-informed neural network technique based on a modified loss function for computational 2{D} and 3{D} solid mechanics.
\newblock {\em Computational Mechanics}, 71(3):543--562, 2023.
\bibitem{ref9}
Ehsan Haghighat, Maziar Raissi, Adrian Moure, Hector Gomez, and Ruben Juanes.
\newblock A physics-informed deep learning framework for inversion and surrogate modeling in solid mechanics.
\newblock {\em Computer Methods in Applied Mechanics and Engineering}, 379:113741, 2021.
\bibitem{ref10}
Enrui Zhang, Ming Dao, George~Em Karniadakis, and Subra Suresh.
\newblock Analyses of internal structures and defects in materials using physics-informed neural networks.
\newblock {\em Science advances}, 8(7):eabk0644, 2022.
\bibitem{ref11}
Shengze Cai, Zhiping Mao, Zhicheng Wang, Minglang Yin, and George~Em Karniadakis.
\newblock Physics-informed neural networks ({PINN}s) for fluid mechanics: A review.
\newblock {\em Acta Mechanica Sinica}, 37(12):1727--1738, 2021.
\bibitem{ref12}
Xiaowei Jin, Shengze Cai, Hui Li, and George~Em Karniadakis.
\newblock {NSF}nets ({N}avier-{S}tokes flow nets): Physics-informed neural networks for the incompressible {N}avier-{S}tokes equations.
\newblock {\em Journal of Computational Physics}, 426:109951, 2021.
\bibitem{ref13}
Ameya~D Jagtap, Zhiping Mao, Nikolaus Adams, and George~Em Karniadakis.
\newblock Physics-informed neural networks for inverse problems in supersonic flows.
\newblock {\em Journal of Computational Physics}, 466:111402, 2022.
\bibitem{ref14}
Jared O'Leary, Joel~A Paulson, and Ali Mesbah.
\newblock Stochastic physics-informed neural ordinary differential equations.
\newblock {\em Journal of Computational Physics}, 468:111466, 2022.
\bibitem{ref15}
Liu Yang, Dongkun Zhang, and George~Em Karniadakis.
\newblock Physics-informed generative adversarial networks for stochastic differential equations.
\newblock {\em SIAM Journal on Scientific Computing}, 42(1):A292--A317, 2020.
\bibitem{ref16}
Yibo Yang and Paris Perdikaris.
\newblock Adversarial uncertainty quantification in physics-informed neural networks.
\newblock {\em Journal of Computational Physics}, 394:136--152, 2019.
\bibitem{ref17}
Liu Yang, Xuhui Meng, and George~Em Karniadakis.
\newblock B-{PINN}s: {B}ayesian physics-informed neural networks for forward and inverse {PDE} problems with noisy data.
\newblock {\em Journal of Computational Physics}, 425:109913, 2021.
\bibitem{ref18}
Sifan Wang, Yujun Teng, and Paris Perdikaris.
\newblock Understanding and mitigating gradient flow pathologies in physics-informed neural networks.
\newblock {\em SIAM Journal on Scientific Computing}, 43(5):A3055--A3081, 2021.
\bibitem{ref19}
Sifan Wang, Xinling Yu, and Paris Perdikaris.
\newblock When and why {PINN}s fail to train: {A} neural tangent kernel perspective.
\newblock {\em Journal of Computational Physics}, 449:110768, 2022.
\bibitem{ref20}
RL~Burden, JD~Faires, and AM~Burden.
\newblock Numerical {A}nalysis (10th).
\newblock {\em Cengage Learning}, 2015.
\bibitem{ref21}
Levi~D McClenny and Ulisses~M Braga-Neto.
\newblock Self-adaptive physics-informed neural networks.
\newblock {\em Journal of Computational Physics}, 474:111722, 2023.
\bibitem{ref22}
Yanjie Song, He~Wang, He~Yang, Maria~Luisa Taccari, and Xiaohui Chen.
\newblock Loss-attentional physics-informed neural networks.
\newblock {\em Journal of Computational Physics}, 501:112781, 2024.
\bibitem{ref23}
Lu~Lu, Xuhui Meng, Zhiping Mao, and George~Em Karniadakis.
\newblock Deep{XDE}: {A} deep learning library for solving differential equations.
\newblock {\em SIAM review}, 63(1):208--228, 2021.
\bibitem{ref24}
Chenxi Wu, Min Zhu, Qinyang Tan, Yadhu Kartha, and Lu~Lu.
\newblock A comprehensive study of non-adaptive and residual-based adaptive sampling for physics-informed neural networks.
\newblock {\em Computer Methods in Applied Mechanics and Engineering}, 403:115671, 2023.
\bibitem{gao2023active}
Wenhan Gao and Chunmei Wang.
\newblock Active learning based sampling for high-dimensional nonlinear partial differential equations.
\newblock {\em Journal of Computational Physics}, 475:111848, 2023.
\bibitem{ref26}
Zhiwei Gao, Liang Yan, and Tao Zhou.
\newblock Failure-informed adaptive sampling for {PINN}s.
\newblock {\em SIAM Journal on Scientific Computing}, 45(4):A1971--A1994, 2023.
\bibitem{ref27}
Sifan Wang, Shyam Sankaran, and Paris Perdikaris.
\newblock Respecting causality for training physics-informed neural networks.
\newblock {\em Computer Methods in Applied Mechanics and Engineering}, 421:116813, 2024.
\bibitem{RRef3}
Arka Daw, Jie Bu, Sifan Wang, Paris Perdikaris, and Anuj Karpatne.
\newblock Mitigating propagation failures in physics-informed neural networks using retain-resample-release ({R}3) sampling.
\newblock In {\em International Conference on Machine Learning}, pages 7264--7302. PMLR, 2023.
\bibitem{ref28}
Ameya~D Jagtap and George~Em Karniadakis.
\newblock Extended physics-informed neural networks ({XPINN}s): A generalized space-time domain decomposition based deep learning framework for nonlinear partial differential equations.
\newblock {\em Communications in Computational Physics}, 28(5), 2020.
\bibitem{ref29}
Ameya~D Jagtap, Ehsan Kharazmi, and George~Em Karniadakis.
\newblock Conservative physics-informed neural networks on discrete domains for conservation laws: {A}pplications to forward and inverse problems.
\newblock {\em Computer Methods in Applied Mechanics and Engineering}, 365:113028, 2020.
\bibitem{ref30}
Khemraj Shukla, Ameya~D Jagtap, and George~Em Karniadakis.
\newblock Parallel physics-informed neural networks via domain decomposition.
\newblock {\em Journal of Computational Physics}, 447:110683, 2021.
\bibitem{ref31}
Chenhao Si and Ming Yan.
\newblock Initialization-enhanced physics-informed neural network with domain decomposition ({IDPINN}).
\newblock {\em Journal of Computational Physics}, 530:113914, 2025.
\bibitem{ref32}
Chuqi Chen, Qixuan Zhou, Yahong Yang, Yang Xiang, and Tao Luo.
\newblock Quantifying training difficulty and accelerating convergence in neural network-based {PDE} solvers.
\newblock {\em arXiv preprint arXiv:2410.06308}, 2024.
\bibitem{ref33}
Jin Song, Ming Zhong, George~Em Karniadakis, and Zhenya Yan.
\newblock Two-stage initial-value iterative physics-informed neural networks for simulating solitary waves of nonlinear wave equations.
\newblock {\em Journal of Computational Physics}, 505:112917, 2024.
\bibitem{RRef5}
Jeremy Yu, Lu~Lu, Xuhui Meng, and George~Em Karniadakis.
\newblock Gradient-enhanced physics-informed neural networks for forward and inverse {PDE} problems.
\newblock {\em Computer Methods in Applied Mechanics and Engineering}, 393:114823, 2022.
\bibitem{liu2024kan}
Ziming Liu, Yixuan Wang, Sachin Vaidya, Fabian Ruehle, James Halverson, Marin Solja{\v{c}}i{\'c}, Thomas~Y Hou, and Max Tegmark.
\newblock {KAN}: {K}olmogorov-{A}rnold {N}etworks.
\newblock {\em arXiv preprint arXiv:2404.19756}, 2024.
\bibitem{shukla2024comprehensive}
Khemraj Shukla, Juan~Diego Toscano, Zhicheng Wang, Zongren Zou, and George~Em Karniadakis.
\newblock A comprehensive and fair comparison between {MLP} and {KAN} representations for differential equations and operator networks.
\newblock {\em Computer Methods in Applied Mechanics and Engineering}, 431:117290, 2024.
\bibitem{wang2025kolmogorov}
Yizheng Wang, Jia Sun, Jinshuai Bai, Cosmin Anitescu, Mohammad~Sadegh Eshaghi, Xiaoying Zhuang, Timon Rabczuk, and Yinghua Liu.
\newblock Kolmogorov--{A}rnold-informed neural network: {A} physics-informed deep learning framework for solving forward and inverse problems based on {K}olmogorov--{A}rnold {N}etworks.
\newblock {\em Computer Methods in Applied Mechanics and Engineering}, 433:117518, 2025.
\bibitem{RRef2}
Zhiyuan Zhao, Xueying Ding, and B~Aditya Prakash.
\newblock Pinnsformer: {A} transformer-based framework for physics-informed neural networks.
\newblock {\em arXiv preprint arXiv:2307.11833}, 2023.
\bibitem{PIRBN}
Jinshuai Bai, Gui-Rong Liu, Ashish Gupta, Laith Alzubaidi, Xi-Qiao Feng, and YuanTong Gu.
\newblock Physics-informed radial basis network ({PIRBN}): A local approximating neural network for solving nonlinear partial differential equations.
\newblock {\em Computer Methods in Applied Mechanics and Engineering}, 415:116290, 2023.
\bibitem{ref34}
Yanzhi Liu, Ruifan Wu, and Ying Jiang.
\newblock Binary structured physics-informed neural networks for solving equations with rapidly changing solutions.
\newblock {\em arXiv preprint arXiv:2401.12806}, 2024.
\bibitem{ref35}
Biao Yuan, He~Wang, Ana Heitor, and Xiaohui Chen.
\newblock f-{PICNN}: A physics-informed convolutional neural network for partial differential equations with space-time domain.
\newblock {\em Journal of Computational Physics}, 515:113284, 2024.
\bibitem{wang2024piratenets}
Sifan Wang, Bowen Li, Yuhan Chen, and Paris Perdikaris.
\newblock {P}iratenets: Physics-informed deep learning with residual adaptive networks.
\newblock {\em Journal of Machine Learning Research}, 25(402):1--51, 2024.
\bibitem{ref36}
Guangtao Zhang, Huiyu Yang, Fang Zhu, Yang Chen, et~al.
\newblock {D}asa-pinns: Differentiable adversarial self-adaptive pointwise weighting scheme for physics-informed neural networks.
\newblock {\em SSRN}, 2023.
\bibitem{ref37}
Zheyuan Hu, Ameya~D Jagtap, George~Em Karniadakis, and Kenji Kawaguchi.
\newblock Augmented {P}hysics-{I}nformed {N}eural {N}etworks ({APINN}s): A gating network-based soft domain decomposition methodology.
\newblock {\em Engineering Applications of Artificial Intelligence}, 126:107183, 2023.
\bibitem{RRef4}
Sokratis~J Anagnostopoulos, Juan~Diego Toscano, Nikolaos Stergiopulos, and George~Em Karniadakis.
\newblock Residual-based attention in physics-informed neural networks.
\newblock {\em Computer Methods in Applied Mechanics and Engineering}, 421:116805, 2024.
\bibitem{XNet1}
Xin Li, Zhihong Xia, and Hongkun Zhang.
\newblock Cauchy activation function and {XN}et.
\newblock {\em Neural Networks}, 188:107375, 2025.
\bibitem{XNet2}
Xin Li, Zhihong~Jeff Xia, and Xiaotao Zheng.
\newblock Model comparisons: {XNet} outperforms {KAN}.
\newblock {\em arXiv preprint arXiv:2410.02033}, 2024.
\bibitem{ref40}
Ziya Uddin, Sai Ganga, Rishi Asthana, and Wubshet Ibrahim.
\newblock Wavelets based physics informed neural networks to solve non-linear differential equations.
\newblock {\em Scientific Reports}, 13(1):2882, 2023.
\bibitem{trefethen2014exponentially}
Lloyd~N Trefethen and JAC Weideman.
\newblock The exponentially convergent trapezoidal rule.
\newblock {\em SIAM review}, 56(3):385--458, 2014.
\bibitem{ranestad2000varieties}
Kristian Ranestad and Frank-Olaf Schreyer.
\newblock Varieties of sums of powers.
\newblock {\em Journal fur die Reine und Angewandte Mathematik}, pages 147--182, 2000.
\bibitem{son2023enhanced}
Hwijae Son, Sung~Woong Cho, and Hyung~Ju Hwang.
\newblock Enhanced physics-informed neural networks with augmented lagrangian relaxation method ({AL}-{PINNs}).
\newblock {\em Neurocomputing}, 548:126424, 2023.
\end{thebibliography}