Ghost Features and Spooky Transfer Learning for Hypercomplex-Valued Neural Networks

Abstract

Hypercomplex numbers extend the concept of complex numbers by introducing additional imaginary components. Besides increasing dimensionality, operations on the imaginary parts provide algebraic and geometrical properties that can be beneficial for solving machine learning problems. In this paper, we show how to create hypercomplex-valued neural network layers where the real part corresponds to the output of a traditional real-valued layer. The additional imaginary parts of these hypercomplex-valued layers produce what we call “ghost features,” which contain enhanced information that is not present in the output of the real-valued layer. Moreover, ghost features can be effectively integrated into a trained neural network through a process we refer to as “spooky transfer learning.” This approach allows us to harness the richness of ghost features, leading to more efficient neural networks. The source code and Jupyter Notebook are available at https://github.com/mevalle/v-nets/.

Index Terms—  Deep learning, hypercomplex-valued neural network, hypercomplex algebra, transfer learning.

1 Introduction

Extending real-valued neural networks to hypercomplex domains, including the complex and quaternionic domains, enables the encoding of multidimensional signals while preserving their intrinsic algebraic and geometric structures [12, 4, 16]. Accordingly, hypercomplex-valued neural networks can compactly and consistently represent correlations, phase relationships, and higher-order interactions among input components [10, 8]. Such representations often enhance generalization and parameter efficiency, particularly in tasks where the underlying data has highly correlated channels or exhibits geometric properties [17].

A key feature of hypercomplex-valued networks is the emergence of latent representations arising from algebraic structures computed over observable input variables, which we refer to as ghost features. Ghost features arise from the algebraic constraints of the hypercomplex algebra, including interactions among coupled components, rather than an explicit architectural design of the network. These can be interpreted as internal modes that are not directly recoverable in the network’s real-valued projection, equivalent to unobservable components in dynamical systems or hidden states in state-space models. The emergence of ghost features is analogous to the Ghost Module mechanism in GhostNets, wherein redundant or low-cost feature maps are generated from intrinsic representations to reduce computational cost while maintaining representational richness [9].

Transfer learning is a machine learning paradigm in which a model trained on a source task is adapted to a related target task [14, 7]. By reusing learned representations, transfer learning can significantly reduce the amount of labeled data and computational resources needed for training in the target domain, accelerate convergence, and enhance generalization. This approach is especially effective when the source and target domains share underlying structural or semantic properties, allowing the model to leverage previously acquired features, invariances, and latent modes. Despite its effectiveness in modern machine learning applications, transfer learning is rarely applied to hypercomplex-valued neural networks due to the scarcity of pre-trained hypercomplex-valued models [5].

This paper formalizes the notion of ghost features in hypercomplex-valued networks, emphasizing how algebra-induced latent structures can be effectively harnessed in pre-training and task adaptation. By investigating the behavior of such features under transfer learning, our work introduces the concept of spooky transfer learning as a method of designing hypercomplex-valued neural networks from trained real-valued models. This paradigm enables pre-trained models to transfer latent structures such as geometric invariances, phase relationships, and higher-order dependencies, to downstream tasks, even when there is weak correspondence between the source and target domains in real-valued terms.

The paper is structured as follows: the next section briefly reviews hypercomplex algebras and hypercomplex-valued neural networks. Ghost features are introduced in Section 3 together with a strategy to construct a hypercomplex-valued layer whose real part corresponds to the output of the given real-valued layer. Section 4 shows how ghost features can be incorporated into a pre-trained neural network by means of the so-called “spooky transfer learning”. An experimental evaluation is conducted in Section 5. The paper finishes with the concluding remarks in Section 6.

2 A Brief Review on Hypercomplex-Valued Neural Networks

This section presents a brief overview of the fundamental concepts of hypercomplex algebras and hypercomplex-valued neural networks. For a more detailed exposition, the reader should refer to [11, 3, 15, 21].

2.1 Hypercomplex Algebra

A hypercomplex number is usually written as

$$x={x}_{0}+{x}_{1}\boldsymbol{i}_{1}+\dots+{x}_{d-1}\boldsymbol{i}_{d-1},$$ (1)

where $x_{0},x_{1},\ldots,x_{d-1}$ are real numbers, and $\boldsymbol{i}_{1},\ldots,\boldsymbol{i}_{d-1}$ are the hyperimaginary units [11]. The set of all hypercomplex numbers defined by (1) is denoted by $\mathbb{H}$. The component $x_{0}$ represents the real part of the hypercomplex number $x$ and can be obtained through the mapping $\mathfrak{Re}:\mathbb{H}\to\mathbb{R}$, $\mathfrak{Re}\!\left\{x\right\}=x_{0}$. The other components of a hypercomplex number can be obtained in a similar fashion using the maps $\mathfrak{Im}_{k}:\mathbb{H}\to\mathbb{R}$, where $\mathfrak{Im}_{k}\!\left\{x\right\}=x_{k}$, for all $k=1,\ldots,d-1$.

A hypercomplex algebra is a vector space $\mathbb{H}$ equipped with a bilinear operation that includes a two-sided identity [16, 11]. In this framework, we define a hypercomplex number as a linear combination of the elements of the canonical basis $\tau=\{1,\boldsymbol{i}_{1},\boldsymbol{i}_{2},\ldots,\boldsymbol{i}_{d-1}\}$, which serves as the basis for $\mathbb{H}$. The coordinate vector of a hypercomplex number $x={x}_{0}+{x}_{1}\boldsymbol{i}_{1}+\dots+{x}_{d-1}\boldsymbol{i}_{d-1}\in\mathbb{H}$ with respect to this canonical basis is obtained through the isomorphism $\varphi:\mathbb{H}\to\mathbb{R}^{d}$ defined by [18]:

$$\varphi(x)=[x_{0},x_{1},\ldots,x_{d-1}]:=\vec{x}\in\mathbb{R}^{d},$$ (2)

A set of hypercomplex numbers inherits the scalar multiplication and the addition from $\mathbb{R}^{d}$. A hypercomplex algebra is obtained by enriching $\mathbb{H}$ with a multiplication operation, which is denoted by “$\times$” and is fully characterized by the products of the basis elements $1,\boldsymbol{i}_{1},\ldots,\boldsymbol{i}_{d-1}$, where the first element $1$ serves as a two-sided identity. Precisely, the product of the hypercomplex units yields hypercomplex numbers

$$\boldsymbol{i}_{\mu}\boldsymbol{i}_{\nu}={p_{\mu\nu}}_{0}+{p_{\mu\nu}}_{1}\boldsymbol{i}_{1}+\dots+{p_{\mu\nu}}_{d-1}\boldsymbol{i}_{d-1},$$ (3)

which form the so-called multiplication table. The multiplication table can be structured as a three-dimensional array of shape $d\times d\times d$ with the coefficients ${p_{\mu\nu}}_{k}$, for $\mu,\nu,k=0,1,\ldots,d-1$. Using the matrices $\mathbf{P}_{0},\ldots,\mathbf{P}_{d-1}$ obtained by slicing the array in the third component and the isomorphism $\varphi$, the product of two hypercomplex numbers $x$ and $y$ is given by

$$x\times y=\big(\vec{x}\mathbf{P}_{0}\vec{y}^{T}\big)+\sum_{k=1}^{d-1}\big(\vec{x}^{T}\mathbf{P}_{k}\vec{y}\big)\boldsymbol{i}_{k}.$$ (4)

Equivalently, the real and imaginary parts of the product satisfy

$$\mathfrak{Re}\!\left\{x\times y\right\}=\vec{x}\mathbf{P}_{0}\vec{y}^{T},$$ (5)

and

$$\mathfrak{Im}_{k}\!\left\{x\times y\right\}=\vec{x}\mathbf{P}_{k}\vec{y}^{T},\quad\forall k=1,\ldots,d-1.$$ (6)

We would like to remark that the product of two hypercomplex numbers can also be expressed as a matrix-vector product, which is particularly interesting from a computational perspective [16, 8, 21].

Finally, a hypercomplex algebra is called non-degenerate with respect to the canonical basis $\tau=\{1,\boldsymbol{i}_{1},\ldots,\boldsymbol{i}_{d-1}\}$ if the matrices $\mathbf{P}_{0},\ldots,\mathbf{P}_{d-1}$ are all non-singular [15, 19]. Non-degenerate hypercomplex algebras guarantee the universal approximation capability of hypercomplex-valued neural networks and are essential for extracting ghost features.

Example 1.

Quaternions, developed by Hamilton in 1843 as an extension of complex numbers, are among the most significant and widely used hypercomplex algebras [12]. The product of quaternions is characterized by the matrices

$$\mathbf{P}_{0}=\begin{bmatrix}1&0&0&0\\ 0&-1&0&0\\ 0&0&-1&0\\ 0&0&0&-1\end{bmatrix},\quad\mathbf{P}_{1}=\begin{bmatrix}0&1&0&0\\ 1&0&0&0\\ 0&0&0&1\\ 0&0&-1&0\end{bmatrix},$$

$$\mathbf{P}_{2}=\begin{bmatrix}0&0&1&0\\ 0&0&0&-1\\ 1&0&0&0\\ 0&1&0&0\end{bmatrix},\quad\text{and}\quad\mathbf{P}_{3}=\begin{bmatrix}0&0&0&1\\ 0&0&1&0\\ 0&-1&0&0\\ 1&0&0&0\end{bmatrix},$$

which are derived straightforwardly from the quaternion multiplication table depicted in Table 1.

Table 1: Quaternion multiplication table.

black!30$\times$	gray!30 $1$	red!30$\boldsymbol{i}$	green!30$\boldsymbol{j}$	blue!30$\boldsymbol{k}$
gray!30$1$	gray!30$1$	red!30$\boldsymbol{i}$	green!30$\boldsymbol{j}$	blue!30$\boldsymbol{k}$
red!30$\boldsymbol{i}$	red!30$\boldsymbol{i}$	gray!30$-1$	blue!30$\boldsymbol{k}$	green!30$-\boldsymbol{j}$
green!30$\boldsymbol{j}$	green!30$\boldsymbol{j}$	blue!30$-\boldsymbol{k}$	gray!30$-1$	red!30$\boldsymbol{i}$
blue!30$\boldsymbol{k}$	blue!30$\boldsymbol{k}$	green!30$\boldsymbol{j}$	red!30$-\boldsymbol{i}$	gray!30$-1$

For example, the real part of the product of $x={x}_{0}+{x}_{1}\boldsymbol{i}+{x}_{2}\boldsymbol{j}+{x}_{3}\boldsymbol{k}$ and $y={y}_{0}+{y}_{1}\boldsymbol{i}+{y}_{2}\boldsymbol{j}+{y}_{3}\boldsymbol{k}$ satisfies

$$\mathfrak{Re}\!\left\{x\times y\right\} =\underbrace{\begin{bmatrix}x_{0},x_{1},x_{2},x_{3}\end{bmatrix}}_{\vec{x}^{T}}\underbrace{\begin{bmatrix}1&0&0&0\\ 0&-1&0&0\\ 0&0&-1&0\\ 0&0&0&-1\end{bmatrix}}_{\mathbf{P}_{0}}\underbrace{\begin{bmatrix}y_{0}\\ y_{1}\\ y_{2}\\ y_{3}\end{bmatrix}}_{\vec{y}}$$

$$=x_{0}y_{0}-x_{1}y_{1}-x_{2}y_{2}-x_{3}y_{3}.$$

while the first imaginary component of the product is given by

$$\mathfrak{Im}_{1}\!\left\{x\times y\right\} =\vec{x}^{T}\mathbf{P}_{1}\vec{y} =x_{0}y_{1}+x_{1}y_{0}+x_{2}y_{3}-x_{3}y_{2}.$$

Since the matrices $\mathbf{P}_{0},\ldots,\mathbf{P}_{3}$ are all non-singular, the quaternions constitute a non-degenerate hypercomplex algebra with respect to the canonical basis $\tau=\{1,\boldsymbol{i},\boldsymbol{j},\boldsymbol{k}\}$.

2.2 Hypercomplex-valued Neural Networks

Hypercomplex-valued neural networks are analogous to traditional models, but with inputs, outputs, and learning parameters (weights and biases) that are hypercomplex numbers rather than real scalars. The basic operations of addition and multiplication by scalar are performed component-wise, and the multiplication is performed using (4). Moreover, a so-called split activation function $\psi:\mathbb{H}\to\mathbb{H}$ is composed of a real-valued activation function $\psi_{\mathbb{R}}:\mathbb{R}\to\mathbb{R}$ applied component-wise to a hypercomplex number $x={x}_{0}+{x}_{1}\boldsymbol{i}_{1}+\dots+{x}_{d-1}\boldsymbol{i}_{d-1}$ as follows:

$$\psi(x)=\psi_{\mathbb{R}}(x_{0})+\psi_{\mathbb{R}}(x_{1})\boldsymbol{i}_{1}+\ldots+\psi_{\mathbb{R}}(x_{d-1})\boldsymbol{i}_{d-1}.$$ (7)

The split-ReLU activation function, obtained by (7) with $\psi_{\mathbb{R}}=\mathtt{relu}$, is commonly used in hypercomplex-valued deep learning models [4, 8, 21, 1]. The following reviews hypercomplex-valued convolutional layers, widely used in deep learning models.

A hypercomplex-valued convolutional layer is a mapping $\mathtt{Conv2d}(\mathbf{W},\boldsymbol{b}):\mathbb{H}^{H\times W\times C}\to\mathbb{H}^{H^{\prime}\times W^{\prime}\times C^{\prime}}$, where the learnable parameteres $\mathbf{W}\in\mathbb{H}^{P\times Q\times C\times C^{\prime}}$ and $\boldsymbol{b}\in\mathbb{H}^{C^{\prime}}$ denote the kernels and the biases, respectively. The pixel value at position $(i,j)$ in the $k$th feature channel of a hypercomplex-valued image $\boldsymbol{X}\in\mathbb{H}^{H\times W\times C}$, where $H$ is the height, $W$ is the width, and $C$ represents the number of hypercomplex-valued channels, is a hypercomplex number denoted as $x_{ijk}={x_{ijk}}_{0}+{x_{ijk}}_{1}\boldsymbol{i}_{1}+\dots+{x_{ijk}}_{d-1}\boldsymbol{i}_{d-1}\in\mathbb{H}$. This hypercomplex number contains multivariate information, such as color data or spectral signatures for hyperspectral images [12, 8, 17, 6]. Given a hypercomplex-valued image $\boldsymbol{X}\in\mathbb{H}^{H\times W\times C}$, a hypercomplex-valued convolutional layer yields a hypercomplex-valued image $\boldsymbol{Y}\in\mathbb{H}^{H^{\prime}\times W^{\prime}\times C^{\prime}}$, with $\boldsymbol{Y}=\mathtt{Conv2d}(\mathbf{W},\boldsymbol{b})(\boldsymbol{X})$, whose entries are given by

$$y_{ijk}=\psi\left(\sum_{p=0}^{P-1}\sum_{q=0}^{Q-1}\sum_{c=0}^{C-1}x_{i^{\prime}j^{\prime}c}\times w_{pqck}+b_{k}\right),$$ (8)

where $i^{\prime}=i+sp$ and $j^{\prime}=j+sq$ are translated pixels locations of the image $\boldsymbol{X}$ with stride $s$, and $\psi:\mathbb{H}\to\mathbb{H}$ denotes the activation function. As with traditional convolutional layers, the shape of $\boldsymbol{Y}$ depends on the padding, which is typically either “same” or “valid” [7].

Like the traditional models, hypercomplex-valued neural networks are universal approximators within a non-degenerate hypercomplex algebra. Specifically, a continuous hypercomplex-valued function $\boldsymbol{f}:\mathbb{H}^{n}\to\mathbb{H}$ can be approximated to any desired precision by a hypercomplex-valued MLP network with a single hidden layer, provided it has a sufficient number of neurons and employs a split continuous non-polynomial activation function. Moreover, this holds if the hypercomplex algebra is non-degenerate with respect to the canonical basis [15]. The approximation capability holds, in particular, for complex-, quaternion-, and Clifford-valued MLP networks that utilize the split-ReLU activation function [1, 2, 19]. Briefly, the resulting hypercomplex-valued MLP model can be derived by applying the approximation theorem of traditional MLP networks to the real and hypercomplex parts of the hypercomplex-valued function $\boldsymbol{f}:\mathbb{H}^{n}\to\mathbb{H}$. The concepts of ghost features and spooky transfer learning discussed in the following are based on this latter remark.

3 Ghost Features

The first layers of a neural network are responsible for extracting informative, structured features necessary for success in machine learning tasks. As pointed out by Han et al. [9], “abundant and even redundant information in the feature maps of well-trained deep neural networks often guarantees a comprehensive understanding of the input data.” Accordingly, in computer vision tasks, similar pairs of feature maps may exist in which one appears to be a ghost of the other. Moreover, the ghost maps can be obtained via simple linear operations and, when coupled with a convolutional layer, can reduce the network’s parameter count and computational complexity [9]. Similarly, by appropriately interpreting a real-valued input as hypercomplex numbers, one can embed a traditional layer within a hypercomplex layer, where the imaginary parts of the output represent ghost features generated by simple linear operations while the real part replicates the original output features. The following details this remark for hypercomplex-valued convolutional layers with split activation function.

Let $\mathbb{H}$ be a hypercomplex algebra with dimension $d$. From (5) and (8), the real part of the $ijk$th entry of the output $\boldsymbol{Y}=\mathtt{Conv2d}(\mathbf{W},\boldsymbol{b})(\boldsymbol{X})$ of a convolutional layer with split activation function satisfies

$$\mathfrak{Re}\!\left\{y_{ijk}\right\}=\psi_{\mathbb{R}}\left(\sum_{p,q,c}\mathfrak{Re}\!\left\{x_{i^{\prime}j^{\prime}c}\times w_{pqck}\right\}+\mathfrak{Re}\!\left\{b_{k}\right\}\right)$$

$$=\psi_{\mathbb{R}}\left(\sum_{p,q,c}\vec{x}_{i^{\prime}j^{\prime}c}\mathbf{P}_{0}\vec{w}_{pqck}+{b_{k}}_{0}\right)$$

$$=\psi_{\mathbb{R}}\left(\sum_{p,q,c}\sum_{\mu=0}^{d-1}x^{(\mathbb{R})}_{i^{\prime}j^{\prime},d(c-1)+\mu}w^{(\mathbb{R})}_{pq,d(c-1)+\mu,k}+{b_{k}}_{0}\right)$$

where $x^{(\mathbb{R})}_{i^{\prime}j^{\prime},d(c-1)+\mu}$ are the entries of a real-valued image $\boldsymbol{X}^{(\mathbb{R})}\in\mathbb{R}^{H\times W\times(dC)}$ obtained by concatenating the real and imaginary parts of the hypercomplex-valued feature channels:

$$\boldsymbol{X}^{(\mathbb{R})}=[\underbrace{{\boldsymbol{X}_{1}}_{0},\ldots,{\boldsymbol{X}_{1}}_{d-1}}_{\begin{subarray}{c}\text{real and imag. parts}\\ \text{of channel \#1}\end{subarray}},\ldots,\underbrace{{\boldsymbol{X}_{C}}_{0},\ldots,{\boldsymbol{X}_{C}}_{d-1}}_{\begin{subarray}{c}\text{real and imag. parts}\\ \text{of channel \#C}\end{subarray}}].$$ (9)

Similarly, $w^{(\mathbb{R})}_{pq,d(c-1)+\mu,k}=\sum_{\nu=0}^{d-1}{p_{\mu\nu}}_{0}{w_{pqck}}_{\nu}$ are the entries of a real-valued convolution filter $\boldsymbol{W}^{(\mathbb{R})}\in\mathbb{R}^{P\times Q\times(dC)\times C^{\prime}}$. Equivalently, using colon notation, we have

$${\boldsymbol{w}^{(\mathbb{R})}_{pq:k}}=[\mathbf{P}_{0}\vec{w}_{pq1k},\ldots,\mathbf{P}_{0}\vec{w}_{pqCk}]\in\mathbb{R}^{dC}.$$ (10)

From the last identities, we conclude that

$$\mathfrak{Re}\!\left\{\mathtt{Conv2D}(\mathbf{W},\boldsymbol{b})(\boldsymbol{X})\right\}=\mathtt{Conv2D}_{\mathbb{R}}\left(\mathbf{W}^{(\mathbb{R})},\boldsymbol{b}^{(\mathbb{R})}\right)\left(\boldsymbol{X}^{(\mathbb{R})}\right),$$ (11)

where $\mathtt{Conv2D}_{\mathbb{R}}(\mathbf{W},\boldsymbol{b}):\mathbb{R}^{W\times H\times(dC)}\to\mathbb{R}^{W^{\prime}\times H^{\prime}\times C^{\prime}}$ is a traditional real-valued convolutional layer applied to the image with the hypercomplex-valued components of the features concatenated. The weights of this layer are given by (10) and the bias vector is given by $\boldsymbol{b}^{(\mathbb{R})}=\mathfrak{Re}\!\left\{\boldsymbol{b}\right\}$, with $\mathfrak{Re}\!\left\{\cdot\right\}$ applied componentwise.

Conversely, given a traditional real-valued convolutional layer with kernel $\mathbf{W}^{(\mathbb{R})}\in\mathbb{R}^{P\times Q\times(dC)\times C^{\prime}}$ and a bias term $\boldsymbol{b}^{(\mathbb{R})}$, the hypercomplex-valued convolutional layer with kernel $\mathbf{W}\in\mathbb{H}^{P\times Q\times C\times C^{\prime}}$ given by

$$w_{pqck}=\varphi^{-1}\left(\mathbf{P}_{0}^{-1}\begin{bmatrix}w^{(\mathbb{R})}_{ij,d(c-1)+1,k}\\ \vdots\\ w^{(\mathbb{R})}_{ij,dc,k}\end{bmatrix}\right)\in\mathbb{H},$$ (12)

and bias $\boldsymbol{b}\in\mathbb{H}^{C^{\prime}}$ such that $\mathfrak{Re}\!\left\{b_{k}\right\}=b_{k}^{(\mathbb{R})}$ for all $k=1,\ldots,C^{\prime}$, satisfies (11), with the hypercomplex-valued image $\boldsymbol{X}\in\mathbb{H}^{H\times W\times C}$ obtained from the input $\boldsymbol{X}^{(\mathbb{R})}\in\mathbb{R}^{H\times W\times(dC)}$ using the inverse isomorphism $\varphi^{-1}$. Furthermore, the imaginary parts

$$\mathfrak{Im}_{k}\!\left\{\mathtt{Conv2d}(\mathbf{W},\boldsymbol{b})(\boldsymbol{X})\right\}\in\mathbb{R}^{H^{\prime}\times W^{\prime}\times C^{\prime}},$$ (13)

are ghost features of the hypercomplex-valued convolutional layer for all $k=1,\ldots,d-1$. The following example illustrates some ghost feature maps of the first convolutional layer of the EfficientNetV2 (B0) pre-trained on ImageNet.

Example 2.

EfficientNets, introduced by Tan and Le in 2019 and further improved in 2021 [13], are highly compact computer vision models designed via a compound scaling factor to uniformly scale network depth, width, and resolution. We considered an EfficientNetV2 (B0) pre-trained on ImageNet and replaced the first real-valued convolutional layer with a quaternion-valued layer, ensuring that (11) holds. Figure 1 shows an input color image and a few features extracted by the first hypercomplex-valued convolutional layer. Precisely, the first column shows the real-valued features equivalent to features yielded by the traditional pre-trained model, while the following columns correspond to the respective ghost features obtained using (13).

Original image

Fig. 1: Image from BloodMNIST [20] followed by real and ghost features obtained from the first convolutional layer of a pre-trained EfficientNetV2 (B0) embedded into quaternions.

4 Spooky Transfer Learning

Transfer learning consists of adapting the feature-extraction and processing structures of a pre-trained network for a new model, thereby leveraging previously trained feature maps to enhance downstream task performance [14, 7]. This paradigm is widely adopted in deep learning applications; however, the limited availability of pre-trained hypercomplex-valued models makes it difficult to effectively leverage transfer learning in hypercomplex domains.

The concept of spooky transfer learning involves embedding a real-valued layer from a pre-trained neural network into a hypercomplex-valued layer. This embedding increases the total number of features, which is mitigated by a dimension reduction layer that selects the most relevant features for the machine learning task. Formally, let $\mathcal{L}_{\mathbb{R}}(\mathbf{W}^{(\mathbb{R})},\mathbf{b}^{(\mathbb{R})})$ denote a layer in a pre-trained deep learning model with parameters $\mathbf{W}^{(\mathbb{R})}$ and $\mathbf{b}^{(\mathbb{R})}$, and let $\mathbb{H}$ be a non-degenerate $d$-dimensional hypercomplex algebra. In analogy to (11), one obtains hypercomplex-valued parameters $\mathbf{W}$ and $\mathbf{b}$ such that:

$$\mathfrak{Re}\!\left\{\mathcal{L}(\mathbf{W},\mathbf{b})(\boldsymbol{X})\right\}=\mathcal{L}_{\mathbb{R}}(\mathbf{W}^{(\mathbb{R})},\mathbf{b}^{(\mathbb{R})})(\boldsymbol{X}^{(\mathbb{R})}),$$ (14)

where the input signal $\boldsymbol{X}$ is obtained from $\boldsymbol{X}^{(\mathbb{R})}$ using $\varphi^{-1}$. From (14), the real part of the hypercomplex-valued layer matches the feature generated by the original layer. Additionally, the $d-1$ imaginary parts of $\mathcal{L}(\mathbf{W},\mathbf{b})(\boldsymbol{X})$ are ghost features yielded by the underlying structure of the product in $\mathbb{H}$. Notably, this output contains $d$ times as many channels as the original layer’s output. In order to mesh the output correctly into the original pipeline, we add a trainable dimension reduction layer, in the form of a dense layer if the original layer was dense, or a convolutional layer with $1\times 1$ kernels if the original layer was convolutional. It is important to highlight that $\mathbf{W}$ and $\mathbf{b}$ are derived from the pre-trained network and are frozen during training, as in traditional transfer learning. In other words, the pre-trained layer produces a non-trainable hypercomplex-valued layer that yields additional ghost features, and a subsequent dimension-reduction layer serves as a trainable layer that combines the original features with the ghost features, thereby enhancing feature extraction.

5 Experimental Evaluation

To showcase the enhanced feature extraction and predictive power achieved by spooky transfer learning, we apply the proposed pipeline to an EfficientNetV2 (B0) [13] for the classification task in the BloodMNIST dataset. MedMNIST-v2 is a large-scale biomedical image dataset for classification tasks [20] containing pre-processed images from several medical specialty domains. BloodMNIST is one such set featuring over 17,000 blood cell microscope images organized into 8 classes, with a provided train-validation-test split of 7:1:2. This dataset was selected as a suitable experiment due to being readily available, broadly documented, and complex enough to warrant the use of deep learning models. Moreover, it is particular to the medical domain, emphasizing the importance of feature extraction and the ability to adapt existing pretrained models by leveraging their representational power.

The baseline comparison used was the available EfficientNetV2 (B0) implementation from the keras.applications library with pre-trained weights on ImageNet. This EfficientNet backbone is followed by a dense (softmax) layer with 8 neurons for classification. For the hypercomplex-valued models, we used the quaternion algebra detailed in Example 1 and applied spooky transfer learning to the first convolutional layer of the EfficientNet, a layer with 32 kernels of size $3\times 3$ and stride 2. The resulting equivalent spooky layer produces an output with 128 feature channels, which is reduced to the original number of features using two methods: a traditional real-valued convolutional layer with 32 $1\times 1$ kernels (indicated by “R”) and a quaternion-valued convolutional layer with 8 $1\times 1$ kernels (indicated by “V”). All models were trained for 15 epochs using the Adam optimizer, cross-entropy loss, and batches of size 32. Table 2 shows the total number of trainable parameters and final test set accuracy per model, while Figure 2 depicts accuracy per training epoch.

Table 2: Traditional vs spooky transfer learning trainable parameters and test set accuracy on BloodMNIST [20].

	Trainable params.	Accuracy (%)
EfficientNetV2 [13]	$10,248$	$95.92\pm{\scriptstyle 0.07}$
Spooky TL (R)	$14,376$	$98.21\pm{\scriptstyle 0.19}$
Spooky TL (V)	$11,304$	$97.49\pm{\scriptstyle 0.51}$

Fig. 2: Test set accuracy per epoch for the baseline EfficientNetV2 (B0) and both Spooky TL variants. The shadowed area indicates the standard deviation.

The results show notable gains in accuracy of the spooky transfer learning-based models over the traditional transfer learning model. Moreover, the added dimension reduction layers contain a small number of trainable parameters, $4128$ and $1056$ for the ‘R’ and ‘V’ variants respectively, thus representing a computationally efficient framework for recombination of features.

6 Concluding Remarks

This paper formalized the concept of ghost features in hypercomplex-valued neural networks, illustrating how they naturally emerge when a traditional layer is embedded into a hypercomplex-valued one. Furthermore, we introduced spooky transfer learning, a novel methodology that embeds pre-trained real-valued layers into hypercomplex frameworks, preserving the original features while inducing ghost features. Experimental results on the BloodMNIST dataset demonstrated the effectiveness of this approach, with spooky transfer learning surpassing traditional transfer learning on the EfficientNetV2 (B0) model. These findings confirm that ghost features provide redundant yet rich, discriminative information that enhances downstream task performance beyond standard real-valued representations. Furthermore, the proposed dimension reduction strategy ensures that these additional features are integrated into existing pipelines with high parameter efficiency. Ultimately, this work provides both a theoretical foundation and a practical path for leveraging pre-trained models and algebraic structures to design more robust deep learning models. A natural next step in this research is the exploration of information theoretic concepts, aimed at quantifying the contribution of ghost features, formally pinpointing its source, and further leveraging it to build lighter models by exploiting representational advantages.