The 2M Multiplication Algorithm for Complex Matrices

The classical approach to multiply complex matrices $A=A_{r}+iA_{i}\in\mathbb C^{m\times k}$ and $B=B_{r}+iB_{i}\in\mathbb C^{k\times n}$ is to expand the product into 4 real matrix multiplications:

Let us call this the 4M algorithm, following [BLISComplex], since it involves 4 real matrix multiplications. 4M multiplication typically forms the backbone of complex floating-point GEMMs in high-performance libraries.

The well-known 3M algorithm reduces this cost to 3 real matrix multiplications, by first computing $A_{r}+A_{i}$ and $B_{r}+B_{i}$:

Note that while (1) is mathematically exact, in floating-point arithmetic the additions $A_{r}+A_{i}$, $B_{r}+B_{i}$ can reduce numerical stability [Higham3M]. Hence, numerical libraries typically offer 3M GEMMs as an opt-in feature.

For scalar complex products, it can be proven that 3 real multiplications is indeed minimal [Winograd2x2]. In this paper, we will show that we can do better for matrix products.

To start, we consider input matrices $A=A_{r}+iA_{i}$, $B=B_{r}+iB_{i}$ with integral real and imaginary parts, and write $AB=C=C_{r}+iC_{i}$. Suppose $m$ is an integer chosen large enough that $\lvert\operatorname{Re}(c_{ij})\rvert,\lvert\operatorname{Im}(c_{ij})\rvert\leq\frac{m}{2}$ for each entry $c_{ij}$ of $C$. Then, if we know $C\bmod m$, we can evidently recover $C$.

Now suppose that $m$ is odd and there is an integer $s$ such that $s^{2}=-1\pmod{m}$, in effect a “real surrogate” for $i$. It can be shown that such an $s$ exists iff $m$ is a product of primes congruent to 1 mod 4. We call such an $m$ a 2M modulus.

Perform the 3M-like preprocessing step

Here $2^{-1}=(m+1)/2$ is the inverse of 2 modulo $m$, explaining why $m$ must be odd. Together, equations (2a–2c) form the 2M multiplication algorithm. As with 3M, the new algorithm requires $\Theta(mn+mk+nk)$ scalar operations in addition to the matrix multiplications.

As a concrete example of Algorithm 1, let us consider complex double-precision GEMM (ZGEMM) with 8-bit integer GEMMs underneath. We enumerate a set of coprime 2M moduli in the range $[1,256]$, chosen using a greedy algorithm:

Recall that the accuracy of an Ozaki-II method is determined primarily by $M$, the product of the chosen moduli. The first $N=16$ moduli of the above sequence, with product $M\approx 2^{117}$, is sufficient in many cases to achieve accuracy comparable to a standard ZGEMM implementation in double-precision arithmetic.

Note that by comparison Ozaki-II real GEMMs, which are not restricted to 2M moduli, have access to a larger set of moduli

Because these moduli decrease less rapidly than 2M moduli, we require only 15 moduli to achieve the same level of accuracy $M\approx 2^{117}$.

The relative sparsity of $\mathcal{M}_{2M}$ w.r.t. $\mathcal{M}_{\text{real}}$ reflects the fact that asymptotically only half of the prime numbers are valid 2M moduli, as observed in Section 3.3. Furthermore, the product of all moduli in $\mathcal{M}_{\text{2M}}$ is $M\approx 2^{152}$, much lower than for real Ozaki-II ($M\approx 2^{342}$). If additional precision is required, e.g. due to a large exponent span in the input matrices [ESC], we may potentially exhaust available 2M moduli. In this case, additional non-2M moduli may be added to the set of moduli using the hybrid 2M/3M algorithm described in Section 3.2.

As mentioned previously, the set of 2M moduli may be varied by discretizing to different imaginary quadratic number rings. Indeed, for 8-bit integer moduli, the Eisenstein integers have a slight advantage over Gaussian integers: we can achieve $M\approx 2^{154.5}$ with 2M moduli. However, this advantage is offset by slightly more complicated quantization/dequantization steps; in particular, quantization involves rounding each point in the complex plane to the nearest point on a hexagonal lattice.

The 2M method also yields computationally efficient methods for the real and complex BLAS-3 rank-$k$ and rank-$2k$ update routines (SYRK, HERK, SYR2K, HER2K). We recall these routines’ definitions, omitting alpha and beta scaling for simplicity:

• •

  SYRK: $C=AA^{T}$,  $A\in\mathbb R^{n\times k}$.

• •

  HERK: $C=AA^{H}$,  $A\in\mathbb C^{n\times k}$.

• •

  SYR2K: $C=AB^{T}+BA^{T}$,  $A,B\in\mathbb R^{n\times k}$.

• •

  HER2K: $C=AB^{H}+BA^{H}$,  $A,B\in\mathbb C^{n\times k}$.

In each case $C$ is (Hermitian) symmetric, so only the lower or upper triangle of $C$ is computed and stored. In theory, the symmetry of $C$ reduces the operation count in half (e.g. $n^{2}k+o(n^{2}k)$ flops for SYRK vs. $2n^{2}k$ for a same-sized GEMM), but library implementations generally cannot obtain the full $2\times$ speedup. The reason is that SYRK-like routines are typically implemented with a modified GEMM kernel that masks its output to the lower or upper triangle of the output matrix $C$ [GotoBLAS3]. Each kernel operates on a fixed-size tile of $C$, leading to wasted computation near the diagonal, where tiles lie partly outside the desired triangle.

With 2M, it is possible to implement symmetric products using full, unmasked GEMMs, while still maintaining the expected 2x speedup. The key idea is that Hermitian symmetry in the problem induces a symmetry relation in the real integer products $C_{\pm}$, allowing us to compute only half as many integer GEMMs. We outline the methods below, which may be applied in either the integer domain (modulo $m$) or in floating point, in conjuction with Ozaki-II.

To demonstrate the efficacy of the 2M method, we present in Figure 1 performance and accuracy results for int8-emulated 2M ZGEMM on a single NVIDIA B200 GPU. All results are with $\alpha=1$, $\beta=0$, and use the Gaussian integer version of 2M.

Figure 1(a) confirms that 2M ZGEMM reaches the expected accuracy results for Ozaki-II. We measure accuracy as the maximum relative error over each entry of $C$, on a test case with $m=n=k=4096$, while varying the number of moduli $N$. Input matrices are chosen using Uchino et al.’s data distribution [Ozaki2] with $\phi=0.5$. Overall, convergence matches the overall trend of real Ozaki-II DGEMM. Note that the choice of complex multiplication method (2M, 3M, or 4M) does not affect accuracy as multiplications are always performed in exact integer arithmetic, and for this reason we do not undertake a detailed accuracy study here, instead referring the reader to [O2Error, ESC, Uchino, Ozaki2].

Figure 1(b) shows 2M ZGEMM performance across a range of square problem sizes, using $N=16$ moduli. Performance is measured in effective teraflops per second, calculated using the floating point operation count from the baseline algorithm: $8mnk$ operations for ZGEMM, $2mnk$ for DGEMM. The modulus count $N$ is chosen to achieve accuracy similar to native-floating point ZGEMM; in fact, Ozaki-II 2M ZGEMM is approximately 10$\times$ more accurate than native floating-point ZGEMM for this input data (cf. Figure 1(a)).

With $N=16$, 2M ZGEMM is up to $8\times$ faster than reference 4M fp64 ZGEMM (cuBLAS 13.1), and 1.4–1.5$\times$ faster than Uchino et al.’s 3M-based GEMMul8 library.⁵⁵5Tested version: https://github.com/RIKEN-RCCS/GEMMul8/commit/3115e709 2M also achieves an average $1.84\times$ speedup in effective TF/s compared to Ozaki-II DGEMM at the same accuracy. That is, in absolute time a single ZGEMM costs approximately $4/1.84\approx 2.16$ DGEMMs, close to theoretical expectations. The cost is not exactly 2 DGEMMs primarily because 2M requires slightly more moduli than the other Ozaki-II methods (16 vs. 15) to achieve a similar accuracy level. Again, this is due to the restricted set of moduli available for 2M.

In Figure 2(a) we fix the problem size to $m=n=k=16384$ and vary $N$ to show the tradeoff between performance and accuracy. 2M shows similar speedups over GEMMul8’s 3M ZGEMM and Ozaki-II DGEMM across the range of $N$, with performance up to 570 TF/s (15$\times$ reference ZGEMM) at $N=8$. Figure 2(b) provides a more detailed heatmap of speedups over reference ZGEMM at a range of sizes and accuracy settings.

With each successive generation of CPU and GPU hardware, narrow-precision GEMM throughput continues to outpace traditional single and double precision arithmetic. The Ozaki-II scheme and other emulation methods take advantage of this trend, using integer quantization and small integer GEMMs to accelerate floating-point GEMMs.

In this paper, we have shown that this integer-quantized approach opens up new algorithms for complex GEMMs that leverage the algebraic structure of quadratic integer rings. In particular, 2M multiplication reduces a complex GEMM to two real GEMMs of the same size, instead of the three or four real GEMMs required by previous algorithms. For rank-$k$ updates, 2M reduces a complex rank-$k$ update ($C=AA^{H}$) to a real GEMM of the same, and a real rank-$k$ update ($C=AA^{T}$) to a real GEMM of half the size. The new 2M algorithm is easy to implement, and achieves practical speedups that closely match these theoretical predictions.