NLI:Non-uniform Linear Interpolation Approximation of Nonlinear Operations for Efficient LLMs Inference
Extracted source text · arXiv 2602.02988
\documentclass{article}
\usepackage{iclr2026_conference,times}
% 可选的数学符号宏集
\input{math_commands.tex}
% ---- common pkgs (精简且无重复) ----
\usepackage{amsmath,amssymb,mathtools,bbm,bm}
\usepackage{graphicx}
\usepackage{booktabs,multirow,makecell}
\usepackage{xcolor,colortbl}
\usepackage{microtype,nicefrac}
\usepackage{wrapfig,adjustbox,gensymb,enumitem}
\usepackage{subfig}
\usepackage{tcolorbox}
\usepackage{xspace}
\usepackage{textcomp}
\usepackage{float}
\usepackage{pifont}
\usepackage{marvosym} % 提供 \Letter 符号
\newcommand{\cmark}{\ding{52}}
\newcommand{\xmark}{\ding{55}}
\let\oldding\ding
\renewcommand{\ding}[2][1]{\scalebox{#1}{\oldding{#2}}}
\usepackage{float} % 为了使用 [H]
\usepackage{subcaption} % 为了 (a)(b) 子标题
\usepackage{booktabs}
% ---- algorithm2e (只保留它) ----
\usepackage[ruled,vlined]{algorithm2e}
\SetKw{KwDownTo}{down to} % 若你的发行版未内置 \KwDownTo，这行可确保可用
% \DontPrintSemicolon % 若不想显示每行末尾的分号，解除注释
% ---- 超链接最后加载 ----
\usepackage{hyperref}
\usepackage{url}
\SetKwProg{Step}{\textbf}{}{} % 定义一个“无前后缀”的块，标题加粗
\SetKw{KwDownTo}{down to} % 若需要 down to
\title{NLI: \underline{N}on-uniform \underline{L}inear \underline{I}nterpolation Approximation of Nonlinear Operations for Efficient LLMs Inference}
% Authors must not appear in the submitted version. They should be hidden
% as long as the \iclrfinalcopy macro remains commented out below.
% Non-anonymous submissions will be rejected without review.
\author{Jiangyong Yu$^{1\ast}$\\
Houmo AI\\
\texttt{jiangyongyufocus@gmail.com}
\And
Xiaomeng Han$^{2\ast}$\\
Southeast University\\
\texttt{mingzhihan7@gmail.com}
\And
Xing Hu$^{1}$\\
Houmo AI\\
\texttt{xing.hu@houmo.ai}
\And
Chen Xu$^{1}$\\
Houmo AI\\
\texttt{chen.xu@houmo.ai}
\And
Zhe Jiang$^{2}$\\
Southeast University\\
\texttt{101013615@seu.edu.cn}
\And
Dawei Yang$^{1\textrm{\Letter}}$\\
Houmo AI\\
\texttt{dawei.yang@houmo.ai}
\thanks{$\ast$ Equal contribution.}
\thanks{$\textrm{\Letter}$ Corresponding author.}
}
% The \author macro works with any number of authors. There are two commands
% used to separate the names and addresses of multiple authors: \And and \AND.
%
% Using \And between authors leaves it to \LaTeX{} to determine where to break
% the lines. Using \AND forces a linebreak at that point. So, if \LaTeX{}
% puts 3 of 4 authors names on the first line, and the last on the second
% line, try using \AND instead of \And before the third author name.
\newcommand{\fix}{\marginpar{FIX}}
\newcommand{\new}{\marginpar{NEW}}
\usepackage{silence}
\WarningFilter{latex}{Command \showhyphens has changed.}
\makeatletter
\providecommand\RSsmallest{7pt} % in case relsize wasn't loaded yet
\renewcommand\RSsmallest{6pt} % must be < 6.94446pt
\makeatother
\iclrfinalcopy % Uncomment for camera-ready version, but NOT for submission.
\begin{document}
\maketitle
\begin{abstract}
Large Language Models (LLMs) have demonstrated remarkable performance across a wide range of tasks, but their deployment is often constrained by substantial memory footprints and computational costs. While prior work has achieved significant progress in compressing and accelerating linear layers, nonlinear layers—such as SiLU, RMSNorm, and Softmax—still heavily depend on high-precision floating-point operations. In this paper, we propose a calibration-free, dynamic-programming-optimal, and hardware-friendly framework called \underline{N}on-uniform \underline{L}inear \underline{I}nterpolation (NLI). NLI is capable of efficiently approximating a variety of nonlinear functions, enabling seamless integration into LLMs and other deep neural networks with almost no loss in accuracy. NLI ingeniously recasts cutpoint selection as a dynamic-programming problem, achieving the \emph{globally} minimal interpolation error in $\mathcal{O}(M \times N^2)$ time via Bellman’s optimality principle. Based on the NLI algorithm, we also design and implement a plug-and-play universal nonlinear computation unit. Hardware experiments demonstrate that the NLI Engine achieves more than 4× improvement in computational efficiency compared to the state-of-the-art designs.
\end{abstract}
\section{Introduction}
Large language models (LLMs; e.g., GPT3-175B \cite{gpt3}, LLaMA3-405B \cite{touvron2023llama}, and Deepseek-R1-671B \cite{guo2024deepseek}) have achieved remarkable success across various domains, such as text translation \cite{translation}, image classification \cite{naeem2023i2mvformer} and text generation \cite{generation}.
However, due to the massive model size, LLMs impose significant demands on both memory bandwidth and computation.
This makes edge deployment of LLMs extremely challenging, hindering the further application of LLMs.
% \begin{wrapfigure}[16]{r}{0.6\textwidth} % 环绕22行, 靠右, 宽度45%
% \vspace{-4mm}
% \centering
% \includegraphics[width=\linewidth]{fig/figure1.pdf}
% \vspace{-5mm}
% \caption{%
% \textbf{(a)}~Range of SiLU activations in representative LLMs; values can exceed $\pm100$ (e.g., Qwen2.5-32B, Qwen3-8B).
% \textbf{(b)}~Wikitext-2 perplexity (log-scale) with FP32, NN-LUT, and our \textbf{NLI}.
% NN-LUT collapses when outliers occur---perplexity skyrockets up to $7.0{\times}10^{4}$---whereas NLI matches FP32 across scales.}
% \label{fig:range_and_compare}
% \vspace{-3mm}
% \end{wrapfigure}
Recent research efforts have focused on using low-bit, high-efficiency data formats \cite{lin2024awq} to improve the computational efficiency of linear layers.
For example, SmoothQuant \cite{xiao2023smoothquant} achieves W8A8 integer quantization by smoothing activation values.
OSTquant \cite{hu2025ostquant} optimizes the distribution of weights and activations through orthogonal transformations and scaling, enabling W4A8 integer quantization.
Moreover, many hardware architectures support low-bit linear computations, further enhancing the efficiency of linear layers.
For example, the Tensor Cores in the NVIDIA H100 \cite{nvidia_h100_tensor_core_gpu} natively support INT8 linear operations.
Gemmini \cite{genc2021gemmini} also supports INT8 linear operations.
% Additionally, the Tensor Cores in the NVIDIA H100 \cite{nvidia_h100_tensor_core_gpu} natively support low-bit linear operations, further enhancing the computational efficiency of linear layers.
However, the computation of nonlinear layers (e.g., Softmax, RMSNorm, SiLU) in LLMs still heavily depends on high-precision floating-point formats (such as FP32), resulting in substantial computational overhead. This further exacerbates the performance disparity between linear and nonlinear operations in LLM inference. For example, in the H100 SXM5, the FP16 linear computational power is 1024 $\times$ greater than that of the special function units, yet in scenarios with a head attention size of 128, the demand for linear computational power is 256 $\times$ that of nonlinear computational power.
\begin{figure}[t]
\centering
\includegraphics[width=1.0\textwidth]{fig/figure1-iclr.pdf}
\caption{
\textbf{(a)}~Range of SiLU activations in representative LLMs; values can exceed $\pm100$ (e.g., Qwen2.5-32B, Qwen3-8B).
\textbf{(b)}~Wikitext-2 perplexity (log-scale) with FP32, NN-LUT, and our \textbf{NLI}.
NN-LUT collapses when outliers occur---perplexity skyrockets up to $7.0{\times}10^{4}$---whereas NLI matches FP32 across scales.}
\label{fig:range_and_compare}
\vspace{-4mm}
\end{figure}
Nonlinear functions in LLMs typically involve hardware-intensive transcendental functions (e.g., exp) and some complicated algebraic functions (e.g., square root, and reciprocal functions).
Some prior works accelerate these functions via hardware-friendly approximations.
For example, Softermax \cite{stevens2021softermax} uses low-precision arithmetic to implement \texttt{Softmax} operations through a base replacement strategy.
I-Bert \cite{kim2021bert} proposed approximation techniques to compute \texttt{GELU}, \texttt{Softmax}, and \texttt{LayerNorm} using INT32 arithmetic.
Although these designs achieve high efficiency and accuracy for certain nonlinear operations in the BERT model, they do not reliably transfer to other LLMs with hundreds of billions of parameters (e.g., LLaMA, Qwen, OPT). Moreover, their inherent hardware inflexibility further limits their applicability in NPUs.
Other researchers have attempted to propose more general-purpose methods.
For example, NN-LUT \cite{yu2022nn} computes nonlinear functions using first-order derivative fitting (${y=sx+t}$), offering good generality within a certain input range.
To maintain model accuracy, it introduces a data calibration method. Although NN-LUT works well for the modest activation spans found in early models such as BERT, its coverage collapses when confronted with the extreme outliers characteristic of LLMs.
As shown in Figure~\ref{fig:range_and_compare}(a), the \texttt{SiLU} inputs of seven representative LLMs frequently exceed $\pm100$. In contrast, NN-LUT was designed and validated only for the input domain $(-5, 5)$, so these inputs clearly lie beyond its expected scope. When the same NN-LUT configuration is applied to these out-of-range values, the approximation error compounds throughout the network and the model fails to converge, producing a Wikitext-2 perplexity surge of up to \(7\times10^{4}\) (Figure~\ref{fig:range_and_compare}\,(b)). Taken together, for LLMs at tens to hundreds of billions of parameters, current acceleration strategies for nonlinear functions remain insufficient. This combination of narrow validity ranges, reliance on calibration, and hardware inflexibility further limits robustness and deployability.
In this paper, we propose a calibration-free, dynamic-programming-optimal, and hardware-friendly framework called \underline{N}on-uniform \underline{L}inear \underline{I}nterpolation (NLI). NLI consists of two parts: NLI-Algorithm (software) and NLI-Engine (hardware). NLI-Algorithm replaces nonlinear evaluations with non-uniform interpolation in the FP16 domain. We formulate the cutpoint selection as a dynamic programming problem with an additive approximation objective that exhibits optimal substructure, enabling solution via the Bellman optimality principle. This yields globally optimal cutpoints and a calibration-free lookup table that is reusable across layers and models. NLI-Engine is a universal plug-and-play hardware block designed for nonlinear function computation based on the NLI algorithm. It improves computational efficiency by optimizing the underlying computation strategy. Software experiments demonstrate that NLI incurs negligible accuracy loss in the open-source LLMs Qwen and LLaMA, nor does it affect the accuracy of other DNN models. Hardware experiments demonstrate that the NLI Engine achieves more than 4× improvement in computational efficiency compared to the state-of-the-art designs.
The main contributions presented are as follows:
\begin{itemize}
\item \textbf{Algorithm:} We introduce the \textbf{N}on-uniform \textbf{L}inear \textbf{I}nterpolation (NLI) framework, which casts cutpoint selection into a dynamic programming (DP) problem. Given a \emph{fixed} nonlinear operator $f$ and a set of $N$ sorted candidate points in the FP16 domain, we minimize an \emph{additive} interpolation error that sums per-segment costs over $M$ segments. Owing to the optimal substructure, the Bellman recursion solves for the \emph{globally optimal} partition in $\mathcal{O}(MN^2)$ time. The resulting LUT is \emph{calibration-free}---it depends only on $f$ and numeric settings rather than data distributions---and is therefore \emph{reusable across layers and models}. We provide a complete pipeline with implementation-ready interfaces and complexity-annotated pseudocode, facilitating immediate deployment with custom CUDA/Triton kernels.
\item \textbf{Hardware Design:} We designed a general-purpose nonlinear computing circuit based on NLI engine. By leveraging a software-hardware co-design approach, we implemented a two-level address translation module to reduce the overhead of address conversion circuits. In addition, we employ pipelining to further boost throughput.
\item \textbf{Comprehensive Experiment:} To demonstrate the practicality of NLI, we conduct experiments from both software and hardware perspectives. The software experiments show that our approximation strategy incurs negligible accuracy degradation in LLMs inference. Moreover, NLI exhibits strong generality, making it applicable not only to LLMs but also to other DNN models.
We further synthesize the NLI engine using the SMIC 28nm technology and conduct a comprehensive analysis of its area, power, throughput and efficiency in comparison with SOTA designs.
\end{itemize}
\section{Background \& Related Work}
\subsection{Nonlinear Operations of LLMs}
Mainstream LLMs such as LLaMA \cite{touvron2023llama} and Qwen \cite{bai2023qwen} are typically composed of multi-head self-attention layers and feed-forward network (FFN) layers.
Each self-attention layer includes one \texttt{Softmax} operation and one \texttt{RMSNorm} operation.
Each FFN layer contains one \texttt{SiLU} operation and one \texttt{RMSNorm} operation.
For MoE-based architectures such as DeepSeek-V3 \cite{guo2024deepseek}, each Transformer block typically contains multiple nonlinear operators—most commonly \texttt{Softmax} and \texttt{RMSNorm}—but can also incorporate \texttt{Sigmoid} for expert routing. Our approach remains broadly applicable here, as the \texttt{Sigmoid} operator can likewise be approximated through non-uniform linear interpolation, thereby offering a unified framework for various activations within MoE architectures.
The definitions of these functions are as follows:
{\small
\begin{equation}
\mathrm{RMSNorm}(x) = \frac{x}{\sqrt{\mathrm{Mean}(x^{2})+\epsilon}}, \quad
\mathrm{Softmax}(x_i) = \frac{e^{\,x_i-\max(x)}}{\sum_{j} e^{\,x_j-\max(x)}}, \quad
\mathrm{SiLU}(x) = \frac{x}{1+e^{-x}}.
\end{equation}
}
\begin{figure}[ht]
\centering
\includegraphics[width=1.0\textwidth]{./fig/figure3_3_iclr.pdf}
\caption{Approximation quality of the SiLU activation over the range $[-150,150]$, which covers $\ge$99.9\% of activations under our measurement protocol (see Figure.~\ref{fig:range_and_compare} (a) and Appendix.~\ref{app:act-measure}). Panel (a) and Panel (b) show the result of NN-LUT, and panel (c) shows our NLI framework. For each method we plot (top-left) the function curve, (bottom-left) the absolute error on a logarithmic scale, and (right) a zoom-in over [-5,5]. Red dots denote LUT cutpoints. NN-LUT suffers from pronounced error spikes, whereas NLI preserves near-machine-precision fidelity throughout the LLM-typical range, reducing worst-case error by several orders of magnitude.} % 图片标题
\label{fig:range} % 图片标签，用于引用
\vspace{-2mm}
\end{figure}
As shown in the equations, these nonlinear operations contribute significantly to the overall runtime due to the high computational cost of floating-point arithmetic.
Therefore, numerous approximation techniques have been proposed to alleviate the hardware costs.
It is worth noting that earlier LLMs, such as OPT \cite{zhang2022opt} and BLOOM \cite{workshop2022bloom}, may have slight variations in their nonlinear operators compared to the equations above.
However, the nonlinear functions they involve are still primarily based on exponential ($e^x$) and square root (${\sqrt{x}}$) operations.
\subsection{General-Purpose Nonlinear Computation Work}
As a general-purpose NPU, it must be capable of handling a wide range of neural networks.
Different types of neural networks utilize various nonlinear operators, such as \texttt{Softmax}, \texttt{Tanh}, and \texttt{ArcTan}. While circuit-level optimization for a specific operator can achieve high efficiency and precision, it lacks flexibility and imposes significant limitations on broader applicability.
Therefore, we attempt to approximate nonlinear functions using a general-purpose approach.
\subsubsection{Linear LUT Fitting}
Linear LUT \cite{cantoni1971optimal, karst1958linear} approximation is a more generalized method for computing nonlinear functions.
It approximates various nonlinear operators by storing N pairs of approximation parameters in a LUT.
The computation formula is as follows:
\begin{equation}
\mathbf{LinearLUT}(x)=
\begin{cases}
k_1 x + b_1 & \text{if } x < d_1, \\
k_i x + b_i & \text{if } d_{i-1} \le x < d_i, \quad \text{for } 1 < i \le N-1, \\
k_N x + b_N & \text{if } x \ge d_{N-1}.
\end{cases}
\label{eq:linear_lut}
\end{equation}
Equation~\ref{eq:linear_lut} shows that, for a fixed number of segments N, the approximation error is governed by the three linear parameters k, b, and d. NN-LUT \cite{yu2022nn} extends LUT-Linear by modelling the search for ($k,b,d$) with a Linear–ReLU–Linear network, whose piece-wise linearity allows the network weights to be analytically converted into the desired parameters. Although this strategy achieves high accuracy within the authors’ test range, our re-implementation (the original code is not public) reveals a strong dependence on the span of the training data, leading to two major drawbacks:
1. \textbf{Severe extrapolation error}. When the training samples cover only a narrow interval—for SiLU, [-10,10]—the network generalises poorly outside that range, and its error explodes (Figure~\ref{fig:range}(a)).
2. \textbf{Poor convergence on wide ranges}. Expanding the interval to [-150,150], which covers 99.9\% of the activations observed in mainstream LLMs, makes optimisation unstable; regions with high curvature (e.g., [-10,10]) cannot be fitted well, as shown in Figure~\ref{fig:range}(b). When the LUTs produced by NN-LUT are used to replace nonlinear operations in LLMs, model accuracy drops sharply (Figure~\ref{fig:range_and_compare}(b)), underscoring that NN-LUT lacks generality for nonlinear functions with wide input ranges.
\subsubsection{Interpolation-Based Approximations and a Unified Error Bound}\label{ibaau}
A standard strategy for approximating a nonlinear operator $f(\cdot)$ on resource-constrained accelerators is to replace it by an \emph{interpolation algorithm} that reconstructs outputs from preselected support points using a local rule. Here, ``interpolation algorithm'' covers linear, piecewise polynomial, and spline-based variants.
\paragraph{Piecewise polynomial and spline-base variants.} Polynomial interpolation and other higher-order \cite{high} interpolation methods have lower computational efficiency, so they are not suitable for NPUs. Previous works, such as NVDLA \cite{nvdla_primer}, have attempted to maintain accuracy by introducing an additional sub-table for regions where the function has steep variations, in addition to the base table.
While this approach improves precision, it requires a two-level computation circuit, leading to lower efficiency. Moreover, for nonlinear functions with smooth slope variations, such as $Cos$, this method fails to achieve high accuracy.
\paragraph{Linear interpolation} For a single segment $[a,b]$ with $f\in C^2([a,b])$, the exact (real-arithmetic) linear interpolant $P(x)$ admits the familiar worst-case bound. However, implementations evaluate a \emph{finite-precision} approximation $\widetilde P(x)$ whose error includes both the analytical interpolation term and a rounding/quantization term. The two can be combined into a single bound:
\begin{equation}
\label{eq:interp-unified}
\max_{x\in[a,b]} \big| f(x)-\widetilde P(x) \big|
\;\le\;
\underbrace{\frac{(b-a)^2}{8}\,\max_{x\in[a,b]} |f''(x)|}_{\text{interpolation term}}
\;+\;
\underbrace{\varepsilon_{\text{num}}}_{\text{finite-precision term}},
\end{equation}
where the first term is the classical linear–interpolation remainder and the finite-precision contribution can be bounded, to first order in the unit roundoffs, by
\begin{equation}
\label{eq:interp-num}
\varepsilon_{\text{num}}
\;\le\;
\Big(\gamma_{1}(u_c)+\gamma_{2}(u_a)\Big)\,\big(|y_i|+|y_{i+1}|\big),
\qquad
\gamma_k(u)=\frac{k\,u}{1-k\,u}.
\end{equation}
Here $y_i=f(a)$ and $y_{i+1}=f(b)$ are the endpoint values for the segment; $u_c$ models coefficient/storage precision (e.g., fp16 vs. fp32 for slopes/LUT values) and $u_a$ models arithmetic precision for the multiply–add evaluation. Equation~\ref{eq:interp-unified} clarifies the accuracy levers of interpolation-based approximations: \emph{(i)} the number of support points (more points shrink the segment length $(b{-}a)$), \emph{(ii)} the distribution of points (allocating denser cuts where $|f''(x)|$ is large tightens the bound), and \emph{(iii)} the storage and arithmetic precision (lower precision increases the total error via the finite-precision term $\varepsilon_{\text{num}}$). In practice, three widely used design heuristics fall short for modern LLMs. First, \textbf{uniform sampling} is intrinsically mismatched to functions with highly nonuniform curvature: since the worst-case linear–interpolation error on $[a,b]$ scales as $\tfrac{(b-a)^2}{8}\max|f''|$, uniform spacing leaves high-curvature regions under-resolved and wastes budget elsewhere; classical approximation theory recommends concentrating nodes in “difficult” regions (e.g., Chebyshev-like layouts) to mitigate Runge-type instability.
Second, \textbf{curvature-driven point allocation} is not universally applicable: for many operators used in inference, the second derivative may be unavailable in closed form, whereas the numerical term $\varepsilon_{\text{num}}$ is typically left uncontrolled.
Third, \textbf{training/calibration–based LUT fitting} ties accuracy to the span and distribution of calibration data; outside that span, extrapolation degrades and convergence becomes brittle.
These limitations motivate a data-free, \emph{globally optimal} placement of cutpoints under a fixed budget.
\section{Software Methodology}
In this section, we present a \emph{calibration-free}, and \emph{globally optimal} (for a fixed knot budget) interval search algorithm for nonlinear functions, together with a more efficient hardware computation strategy.
\begin{figure}
\centering
\vspace{-2pt}
\includegraphics[width=1.0\textwidth]{fig/figure4_iclr.pdf}
\caption{Comparison between the full-precision reference and our NLI approximation on two representative nonlinear functions—\texttt{exp} and \texttt{rsqrt}.
We use 10 macro-intervals: the first and last are not subdivided, and each of the middle eight is uniformly partitioned into 32 bins; accounting for shared endpoints, this yields $2 + 32\times 8 + 1 = 259$ cutpoints.
The lower panels plot absolute error (log scale); the worst-case error stays below $1.2\times 10^{-3}$ across the FP16 domain.
Additional visualizations and cutpoint layouts are provided in Appendix~\ref{appaddnul}.}
\label{fig:fitting}
\vspace{-2mm}
\end{figure}
\subsection{Proposed Non-Uniform Interpolation LUT via Dynamic Programming}
The discussion in Sec~\ref{ibaau} (Equation.~\ref{eq:interp-unified}) shows that linear–interpolation error reflects both curvature and finite–precision terms, which explains why \emph{uniform sampling}, \emph{curvature–driven allocation}, and \emph{training/calibration–based LUTs} each break down on LLMs. We therefore tackle the complementary design question: given a fixed budget of $M$ cutpoints (i.e., $M\!-\!1$ segments), where should the cutpoints be placed on the FP16 grid to minimize the global error actually seen by hardware? We cast this as a discrete dynamic program over the FP16 domain: the DP states and transitions (defined below) minimize the \emph{mean relative error} on $\{x_0,\dots,x_{N-1}\}$, include \emph{endpoint clamping at both the first and last cutpoints} (the left clamp is captured in the $D[0,k]$ boundary term, and the right clamp as an explicit tail term), and are \emph{calibration–free}.
\paragraph{Setup.}
We implement nonlinear operators via table lookup on the FP16 grid.
Let $\mathcal{X}=\{x_0<\cdots<x_{N-1}\}$ be all \emph{finite} FP16 numbers that lie in the legal domain of $f$ (NaNs are dropped; $\pm\infty$ are clamped to the nearest finite endpoint).
We choose $M$ cutpoints $\mathcal{B}=\{b_0,\dots,b_{M-1}\}\subset\mathcal{X}$ with $b_0=x_0$ and $b_{M-1}=x_{N-1}$, inducing $M\!-\!1$ macro-intervals.
At inference time, inputs below $b_0$ (resp. above $b_{M-1}$) are clamped to $b_0$ (resp. $b_{M-1}$).
\noindent\textbf{Problem statement.}
Given $f$, we seek $\mathcal{B}$ that minimizes the average \emph{relative} interpolation error on the FP16 grid.
Within each $[b_i,b_{i+1}]$ we approximate $f$ by the straight line through the \emph{endpoints} $(b_i,f(b_i))$ and $(b_{i+1},f(b_{i+1}))$.
\paragraph{Rationale.}
We optimize $M$ cutpoints over the sorted FP16 grid $\{x_0,\dots,x_{N-1}\}$ for a target function $f$.
Define two DP tables with explicit shapes and index ranges:
\[
D\in\mathbb{R}^{M\times N},\quad
P\in\mathbb{Z}^{M\times N},\qquad
L\in\{0,\dots,M{-}1\},\ k\in\{0,\dots,N{-}1\}.
\]
Their meanings are:
\begin{itemize}[leftmargin=1.2em, itemsep=0pt]
\item $D[L,k]$: the minimum error over the prefix $\{x_0,\dots,x_k\}$ when $x_k$ is chosen as the $L$-th cutpoint.
\item $P[L,k]$: the predecessor index (location of the $(L\!-\!1)$-th cutpoint) that attains $D[L,k]$.
\end{itemize}
\noindent\textbf{Error functional (mean relative error).}
For any segment $[x_i,x_k]$ ($i<k$), let $P_{i,k}(x)$ be the straight line through endpoints $(x_i,f(x_i))$ and $(x_k,f(x_k))$.
We define
\[
\mathrm{Err}(i\!\rightarrow\!k)
= \frac{1}{k-i+1}\sum_{j=i}^{k}
\frac{\bigl|f(x_j)-P_{i,k}(x_j)\bigr|}
{\max\{|f(x_j)|,\tau\}},
\]
where we set $\tau=2^{-14}$, which equals the \emph{smallest positive normal} value in
IEEE~754 binary16 (FP16).
This choice avoids over-amplifying relative errors for numerically near-zero activations while aligning the denominator floor with the FP16 normal/subnormal boundary.
\noindent\textbf{Boundary.}
Inputs smaller than the first cutpoint are replaced by the first cutpoint’s value.
Hence, for the first cutpoint placed at $x_k$,
\[
D[0,k] \;=\; \frac{1}{k+1}\sum_{j=0}^{k}
\frac{\bigl|f(x_j)-f(x_k)\bigr|}{\max\{|f(x_j)|,\tau\}},
\qquad
P[0,k] \;=\; k.
\]
\noindent\textbf{Transition.}
For $1\!\le\!L\!\le\!M\!-\!1$ and $L\!\le\!k\!\le\!N\!-\!1$,
\[
D[L,k] \;=\;
\min_{\,i\in\{L-1,\dots,k-1\}}\;
\Big\{\, D[L-1,i] \;+\; \mathrm{Err}(i\!\rightarrow\!k) \;+\; \mathrm{last\_error}(L,k) \,\Big\},
\]
where the tail-clamping penalty is nonzero \emph{only} at the last cutpoint:
\[
\mathrm{last\_error}(L,k)=
\begin{cases}
\displaystyle
\frac{1}{\max\{1,\,N-1-k\}}\sum_{j=k+1}^{N-1}
\frac{\bigl|f(x_j)-f(x_k)\bigr|}{\max\{|f(x_j)|,\tau\}}, & L=M-1,\\[8pt]
0, & \text{otherwise.}
\end{cases}
\]
Set $P[L,k]$ to the argmin index $i$ that achieves $D[L,k]$.
\noindent\textbf{Optimal value.}
The best $M$ cutpoints correspond to the minimum value in the last DP row:
\[
\text{Cost}^\ast \;=\; \min_{k} D[M-1,k].
\]
\vspace{-3mm}
\noindent\textbf{Backtracking.}
Let $k^\star=\arg\min_{k} D[M-1,k]$.
Recover the indices of cutpoints by following predecessors:
\[
\text{best\_points} = \bigl[k^\star,\; P[M-1,k^\star],\; P[M-2,P[M-1,k^\star]],\;\dots\bigr] \text{ (then reverse to ascending).}
\]
\paragraph{Computational cost.}
The straightforward implementation is $\mathcal{O}(M \times N^2)$ (each state scans all predecessors). For typical settings $(M\le11, N\le63\,488)$ the search completes in under ten minutes on a single NVIDIA RTX 4090 GPU with Triton.
The complete procedure is summarized in Algorithm~\ref{alg:nli-dp}.
A naïve variant could set the DP budget to the \emph{full} fine-grained table and search over $M{=}259$ cutpoints directly, but this inflates the DP time roughly $26\times$ (empirically $\approx$ 5 hours on our setup) and would also mandate a large number of comparators in hardware, hurting throughput, area, and power.
Instead, we adopt a hardware-consistent layout with \emph{ten} macro-intervals: the first and last are not subdivided, and each of the middle eight is uniformly partitioned into 32 bins.
Under this layout, the DP only needs to optimize the \emph{macro} endpoints, i.e., $M{=}11$ cutpoints, which reduces search time by about $26\times$ while producing LUTs that map directly to the two-level address translation.
This design yields higher hardware efficiency (fewer comparators and smaller on-chip tables). Figure~\ref{fig:range}(c) and Figure~\ref{fig:fitting} further visualise the resulting approximation quality: with only $2{+}8\times32{+}1 = 259$ cutpoints, our NLI overlaps almost perfectly with the FP32 curve, keeping the worst-case absolute error below $1.2\times10^{-3}$.
Additional operators and cutpoint configurations are reported in Appendix~\ref{appaddnul}.
\subsection{Hardware-Friendly Computation Strategy}
\label{frendly}
In this section, we propose a hardware-friendly computation strategy that utilizes two-level address translation.
By leveraging simple computations, this approach significantly reduces the number of comparators and improves hardware efficiency.
Traditional address translation modules rely on multiple comparators. For example, with $259$ cut points, the system generates $258$ sub-intervals, requiring $259$ parallel comparisons to determine the corresponding LUT address for an input data point.
This leads to significant hardware overhead.
To address this, we adopt a two-level address translation strategy.
First, we divide the 259 cut points into a structure of ($2 + 8 \times 32 + 1$), meaning 8 sub-intervals, each containing 32 cut points, along with two boundary values representing positive and negative infinity.
Then, we use 10 comparators to determine which major interval the input data belongs to.
Next, based on the interval index, we retrieve the multiply scale factor for that interval.
After performing the multiplication, we apply a floor operation, and the resulting integer value serves as the LUT index.
By utilizing a pipelined design, the latency introduced by the two-level approach is effectively hidden.
Additionally, all multiply scale factors are precomputed offline and preloaded into dedicated registers. In this case, only 10 16-bit registers are required for storage.
The detailed steps are shown in \textcolor{blue}{Algorithm~\ref{alg:nli-dp}}.
As shown in Algorithm \ref{alg:nli_forward}, the two-level address translation approach utilizes simple lookup and arithmetic operations. By applying the transformation $y = kx$ within each sub-interval, the address conversion is efficiently completed, eliminating the need for over 200 FP16 comparators.
The computation formula for the linear interpolation can be summarized as follows:
\begingroup
\fontsize{8pt}{9.6pt}\selectfont
\begin{equation}
y=Decimal\times(LUT[Index + 1]-LUT[Index])+LUT[Index]
\end{equation}
\endgroup
\section{Hardware Methodology}
\begin{figure*}[ht]
\centering
\includegraphics[width=1.0\textwidth]{fig/723version.pdf}
\caption{The hardware circuit design of a nonlinear computing unit, using two computing circuits sharing a set of Multiply scale factor and function value registers as an example.} % 图片标题
\label{fig:example} % 图片标签，用于引用
\vspace{-4mm}
\end{figure*}
In this section, we introduce the NLI Engine, a plug-and-play hardware module that leverages the optimized computation flow of the NLI algorithm to enable efficient nonlinear computation on NPUs.
Since current DNN accelerators fabricated with 28nm technology, e.g., Google TPU v1 \cite{tpu_transformation}, operate at around 1GHz, we have adopted a four-stage pipeline design to align with the clock frequencies of most NPUs.
The NLI engine implements the following functions: LUT and interval cutpoint preloading, two-level address translation, interpolation coefficient (Decimal) computation, and linear interpolation algorithm.
The following are the detailed hardware design specifications of NLI engine:
% \begin{figure*}[ht]
% \centering
% \includegraphics[width=1\textwidth]{NLI/fig/hardopt.pdf} % 替换example.pdf为你的PDF图片文件名
% \caption{(a) illustrates the data distribution after the $Q\times K^T$ operation in the first layer of the LLaMA-1B model. (b) presents the sparsity-optimized circuit, using a 2:1 ratio of comparator units to computational units as an example.} % 图片标题
% \label{fig:hardopt} % 图片标签，用于引用
% \vspace{-8pt}
% \end{figure*}
\textbf{Stage 1 — Major-interval select \& alignment.}
An interval comparator (10 comparators) selects the macro-interval index $I\!\in\!\{0,\dots,9\}$ using preloaded left boundaries $\texttt{left}[I]$; inputs are clamped to $[\texttt{left}[0],\texttt{left}[9]]$.
An FP16 subtractor computes the aligned offset $\Delta x = x - \texttt{left}[I]$.
\textbf{Stage 2 — Micro-address generation.}
Each macro-interval has a preloaded scale $\texttt{mul}[I]$ and a base pointer $\texttt{base}[I]$ (start index in the global LUT).
An FP16 multiplier forms $u=\Delta x\cdot\texttt{mul}[I]$; the floor unit yields the integer micro-index $a=\lfloor u\rfloor$ and the fractional coefficient $t=u-a$ ($a\in[0,31]$ for the middle eight intervals, $a=0$ at the ends).
The global LUT address is $g=\texttt{base}[I]+a$.
\textbf{Stage 3 — Table read \& slope prep.}
A dual-port SRAM (259 entries) returns two adjacent values:
$y_0=\texttt{LUT}[g]$ and $y_1=\texttt{LUT}[g+1]$ in one cycle.
An FP16 subtractor computes the local slope $\Delta y = y_1 - y_0$.
\textbf{Stage 4 — Linear interpolation.}
An FP16 multiplier–adder evaluates $y = y_0 + t\cdot\Delta y$ (optionally fused FMA), then rounds to FP16.
This four-stage pipeline, together with the two-level address translation ($I,a$), reduces comparators from $259$ to $10$ and shrinks address-translation overhead, improving throughput, area, and power.
\section{Evaluation}
\begin{table}[ht]
\tiny
\renewcommand{\arraystretch}{0.4}
\centering
\caption{NLI accuracy across datasets. Columns are datasets (higher is better except perplexity). Boldface rows highlight our method.}
\resizebox{\textwidth}{!}{
\begin{tabular}{p{2.8cm}lcccccc}
\toprule
\multirow{2}{*}{Model} & \multirow{2}{*}{Method}
& \multicolumn{4}{c}{Accuracy (↑)}
& \multicolumn{1}{c}{Perplexity (↓)} \\
\cmidrule(lr){3-6}\cmidrule(l){7-7}
& & MMLU & GSM8k & HumanEval & Zero-shot Avg & Wikitext-2 \\
\midrule
\multirow{3}{*}{Llama3-8B}
& FP32 & 62.16 & 50.19 & 35.37 & 68.11 & 6.14 \\
& NN-LUT & 60.01 & 49.42 & 34.15 & 65.93 & 8.28 \\
\rowcolor{gray!25}
\cellcolor{white} & \textbf{NLI} & \textbf{62.14} & \textbf{50.49} & \textbf{35.37} & \textbf{68.24} & \textbf{6.14} \\
\midrule
\multirow{3}{*}{Llama3-70B}
& FP32 & 75.13 & 80.82 & 40.85 & 73.78 & 2.86 \\
& NN-LUT & 73.99 & 79.06 & 37.2 & 72.48 & 5.13 \\
\rowcolor{gray!25}
\cellcolor{white} & \textbf{NLI} & \textbf{75.11} & \textbf{81.27} & \textbf{40.63} & \textbf{73.85} & \textbf{2.86} \\
\midrule
\multirow{3}{*}{Qwen2.5-7B}
& FP32 & 70.56 & 44.28 & 40.24 & 67.48 & 7.46 \\
& NN-LUT & 25.51 & 0 & 0 & 30.13 & 28194 \\
\rowcolor{gray!25}
\cellcolor{white} & \textbf{NLI} & \textbf{70.67} & \textbf{43.97} & \textbf{39.63} & \textbf{67.63} & \textbf{7.46} \\
\midrule
\multirow{3}{*}{Qwen2.5-32B}
& FP32 & 81.74 & 70.13 & 56.71 & 70.76 & 5.32 \\
& NN-LUT & 25.51 & 0 & 0 & 30.70 & 70360 \\
\rowcolor{gray!25}
\cellcolor{white} & \textbf{NLI} & \textbf{81.68} & \textbf{70.07} & \textbf{55.88} & \textbf{70.67} & \textbf{5.32} \\
\midrule
\multirow{3}{*}{Qwen1.5-110B}
& FP32 & 79.26 & 84.44 & 50.84 & 72.42 & 4.81 \\
& NN-LUT & 75.99 & 76.15 & 43.03 & 69.08 & 6.83 \\
\rowcolor{gray!25}
\cellcolor{white} & \textbf{NLI} & \textbf{79.31} & \textbf{84.41} & \textbf{50.16} & \textbf{72.48} & \textbf{4.81} \\
\midrule
\multirow{3}{*}{Qwen3-8B}
& FP32 & 72.94 & 88.10 & 63.41 & 66.68 & 9.72 \\
& NN-LUT & 23.59 & 0 & 0 & 33.61 & 825.31 \\
\rowcolor{gray!25}
\cellcolor{white} & \textbf{NLI} & \textbf{72.98} & \textbf{88.17} & \textbf{62.59} & \textbf{66.76} & \textbf{9.73} \\
\midrule
\multirow{3}{*}{Qwen3-30B-A3B}
& FP32 & 77.86 & 85.44 & 31.21 & 67.59 & 8.70 \\
& NN-LUT & 28.46 & 0 & 0 & 65.60 & 10.76 \\
\rowcolor{gray!25}
\cellcolor{white} & \textbf{NLI} & \textbf{77.88} & \textbf{85.39} & \textbf{30.38} & \textbf{67.65} & \textbf{8.70} \\
\bottomrule
\end{tabular}
}
\label{t1}
\vspace{-3mm}
\end{table}
\vspace{-2mm}
\subsection{Software Evaluation}
For evaluation, we first run NLI to search macro cutpoints for each nonlinear operator under a 10–macro-interval layout: the middle eight intervals are uniformly split into 32 bins, while the first/last are unsplit with endpoint clamping.
We then replace nonlinear operators in Llama and Qwen (PyTorch~\cite{paszke2019pytorch}) and report perplexity on Wikitext-2~\cite{merity2016wiki}, a standard zero-shot suite (ARC-c/e~\cite{clark2018think}, BoolQ~\cite{clark2019boolq}, PIQA~\cite{bisk2020piqa}, HellaSwag~\cite{zellers2019hellaswag}, OBQA~\cite{mihaylov2018obqa}, LAMBADA~\cite{paperno2016lambada}, SIQA~\cite{sap2019siqa}, WinoGrande~\cite{sakaguchi2021winogrande}), and three widely used benchmarks—MMLU~\cite{hendrycks2020measuring} (broad factual/problem-solving across 57 disciplines), HumanEval~\cite{chen2021evaluating} (functional code-generation accuracy), and GSM8k~\cite{cobbe2021training} (multi-step grade-school math).
We cover multiple model scales and, to test generality beyond LLMs, repeat the substitution on ViT and CNNs.
From Table~\ref{t1}, we observe that replacing nonlinear operators with NLI in Llama and Qwen yields \emph{no accuracy drop} on the zero-shot suite (detailed results in Appendix~\ref{app:full-zeroshot}) and does not increase PPL.
Moreover, performance on MMLU, HumanEval, and GSM8k remains nearly on par with FP32.
Our nonlinear computation strategy has minimal impact on the accuracy of open-source Llama/Qwen models—even without any data calibration. Beyond LLMs, substituting NLI for nonlinear operators in ViT and representative CNNs yields no statistically significant accuracy degradation; full per-model results are reported in Appendix~\ref{app:vision} (Table~\ref{tab:nli_models}).
\vspace{-2mm}
\subsubsection{Ablation}
\begin{table}[ht]
\centering
% 左表
\begin{minipage}[t]{0.46\linewidth}
\centering
\caption{Ablation I on Qwen2.5-7B: two-level NLI (259) vs macro-only (11).}
\label{tab:ab1}
\resizebox{\linewidth}{!}{
\begin{tabular}{l r cc}
\toprule
Method & Cutpoints & MMLU & GSM8k \\
\midrule
FP32 & -- & 70.56 & 44.28 \\
NLI (2+8$\times$32+1) & 259 & \textbf{70.67} & \textbf{43.97} \\
Macro-only (DP, $M{=}11$) & 11 & 21.14 & 0 \\
\bottomrule
\end{tabular}
}
\end{minipage}\hfill
% 右表
\begin{minipage}[t]{0.52\linewidth}
\centering
\caption{Ablation II on Qwen2.5-7B: accuracy (↑) and cutpoint-search time (↓).}
\label{tab:ab2}
\resizebox{\linewidth}{!}{
\begin{tabular}{l r c c c}
\toprule
Method & Cutpoints & MMLU & GSM8k & Search time(s) \\
\midrule
FP32 & -- & 70.56 & 44.28 & -- \\
NLI (2+8$\times$32+1) & 259 & \textbf{70.67} & 43.97 & 610 \\
Non-uniform 259 (DP) & 259 & 70.65 & \textbf{44.08} & 17000 \\
\bottomrule
\end{tabular}
}
\end{minipage}
\vspace{-5mm}
\end{table}
As shown in Table~\ref{tab:ab1}, we evaluate on Qwen2.5-7B with zero-shot MMLU and GSM8K.
Compared to the proposed two-level layout \textbf{NLI} (259 total cutpoints), the \emph{macro-only} variant
optimizes only $M{=}11$ endpoints and applies a single piecewise-linear fit over the FP16 grid
(with no per-macro uniform sub-bins).
This ablation isolates the benefit of uniform micro-partitioning within each macro-interval—
demonstrating that using only 11 cutpoints is insufficient on both benchmarks.
We compare \textbf{NLI} with a direct DP over \textbf{259 non-uniform} cutpoints (no macro/micro constraint) on \textbf{Qwen2.5-7B}. As shown in Table~\ref{tab:ab2}, accuracy on MMLU/GSM8k is essentially unchanged, while the search time explodes (\(\sim\!28\times\) slower) and the resulting layout is hardware-unfriendly, incurring higher area/latency costs.
\begin{wraptable}[7]{r}{5.5cm}
\centering
\small
\renewcommand{\arraystretch}{0.5}
\caption{Ablation III on Qwen2.5-7B: accuracy comparison on MMLU and GSM8k.}
\label{tab:ab3}
\resizebox{1.0\linewidth}{!}{
\begin{tabular}{l r cc}
\toprule
Method & Cutpoints & MMLU & GSM8k \\
\midrule
FP32 & -- & 70.56 & 44.28 \\
NLI (2+8$\times$32+1) & 259 & \textbf{70.65} & \textbf{43.97} \\
Uniform 259 & 259 & 45.91 & 18.13 \\
Curvature 259 & 259 & 65.74 & 32.58 \\
\bottomrule
\end{tabular}}
\end{wraptable}
We further compare \textbf{NLI} with two common heuristics using the same total budget of 259 points: (i) \textbf{Uniform 259} (uniformly spaced over the FP16 grid), and (ii) \textbf{Curvature 259} (density proportional to a curvature proxy). Both use the same linear interpolation and inference pipeline as NLI. Table~\ref{tab:ab3} summarizes results.
\vspace{-2mm}
\subsection{Hardware Evaluation}
The experiments in the previous sections have demonstrated that our NLI approximation method achieves high accuracy and generality.
In this section, to demonstrate the efficiency of the NLI engine, we compare NLI engine with two state-of-the-art nonlinear computational units.
% In this section, to validate the efficiency of our basic hardware design, we evaluate NLI by implementing custom hardware and analyzing the corresponding nonlinear hardware overhead based on the throughput of existing models.
We implemented the hardware circuit design using Chisel \cite{bachrach2012chisel}. The circuit was synthesized with Design Compiler \cite{muchnick1997dc} under the SMIC 28nm process library to obtain area , power, and timing information.
% Power analysis was conducted using PT-PX \cite{prime_time_px_user_guide}.
% \begin{wraptable}[6]{r}{0.45\textwidth}
% \caption{Hardware area breakdown}
% \begin{adjustbox}{max width=\linewidth}
% \begin{tabular}{@{}lrrrrrc@{}}
% \toprule
% & LUT & Comparator & Multiplier & Adder & Others & Total$(\mu m^2)$ \\
% \midrule
% NN-LUT & 12268 & 10496 & 205 & 134 & 135 & 23238 \\
% RI-LUT & 12268 & 10496 & 410 & 268 & 205 & 23647 \\
% \textbf{NLI} & \textbf{6224} & \textbf{410} &
% \textbf{205} & \textbf{536} & \textbf{191} & \textbf{7566} \\
% \bottomrule
% \end{tabular}
% \end{adjustbox}
% \label{tab:resources}
% \end{wraptable}
\begin{table}[ht]
\centering
\begin{minipage}[t]{0.48\linewidth}
\centering
\caption{Hardware area breakdown.}
\label{tab:resources}
\begin{adjustbox}{max width=\linewidth}
\begin{tabular}{@{}lrrrrrc@{}}
\toprule
& LUT & Comparator & Multiplier & Adder & Others & Total $(\mu m^2)$ \\
\midrule
NN-LUT & 12268 & 10496 & 205 & 134 & 135 & 23238 \\
RI-LUT & 12268 & 10496 & 410 & 268 & 205 & 23647 \\
\textbf{NLI} & \textbf{6445} & \textbf{410} &
\textbf{205} & \textbf{536} & \textbf{191} & \textbf{7787} \\
\bottomrule
\end{tabular}
\end{adjustbox}
\end{minipage}\hfill
\begin{minipage}[t]{0.48\linewidth}
\centering
\caption{Hardware comparison.}
\label{tab:eff}
\footnotesize
\begin{adjustbox}{max width=\linewidth}
\begin{tabular}{@{}lrrrrr@{}}
\toprule
& Clock Freq. & Area $(\mu m^2)$ & Power $(mW)$ & Throughput & Efficiency \\
\midrule
NN-LUT & 1GHz & 23238 & 46 & 1G & 0.94 \\
RI-LUT & 1GHz & 23647 & 48 & 1G & 0.88 \\
\textbf{NLI} & \textbf{1GHz} & \textbf{7787} & \textbf{34} &
\textbf{1G} & \textbf{3.78} \\
\bottomrule
\end{tabular}
\end{adjustbox}
\end{minipage}
\vspace{-3mm}
\end{table}
Table \ref{tab:resources} shows the areas of LUTs, multipliers, adders, and Others (registers, shifters, etc.) in the three general nonlinear computational units: NLI, NN-LUT, and RI-LUT. It can be seen that NLI (with 259 cut points) saves 68\% and 69\% in area compared with the other two SOTAs (NN-LUT and RI-LUT, each with 256 cut points).
NN-LUT~\cite{yu2022nn} and RI-LUT~\cite{kim2023range} require storing $512 \times 16 $ bits of data (256 K values and 256 B values). NLI only needs to store data for 259 16-bit cut points and 10 16-bit scale factors , so it has a smaller LUT area. Also, our hardware-friendly algorithm cuts down the number of comparators. NN-LUT and RI-LUT need 256 comparators to divide into 256 interval addresses, while NLI only needs 10 comparators for ten intervals.
% \begin{wraptable}[8]{r}{0.5\textwidth}
% \caption{Hardware comparison}
% \footnotesize
% \begin{adjustbox}{max width=\linewidth}
% \begin{tabular}{@{}lrrrrr@{}}
% \toprule
% &Clock Frequency& Area $(\mu m^2)$ & Power $(mW)$ & Throughput & Efficiency \\
% \midrule
% NN-LUT &1GHz & 23238 & 46 & 1G & 0.94 \\
% RI-LUT &1GHz & 23647 & 48 & 1G & 0.88 \\
% \textbf{NLI} &\textbf{1GHz}& \textbf{7566} & \textbf{34} &
% \textbf{1G} & \textbf{3.89} \\
% \bottomrule
% \end{tabular}
% \end{adjustbox}
% \label{tab:eff}
% \end{wraptable}
Table \ref{tab:eff} provides a detailed comparison of the NLI Engine, NN-LUT, and RI-LUT hardware modules under a 1 GHz clock frequency in terms of area, power, Throughput ( the number of nonlinear operators computed per cycle $\times$ clock frequency), and Efficiency ($Throughput/ (area \times power)$).
The NLI Engine exhibits lower power consumption, primarily due to reduced static power resulting from fewer LUTs and comparator modules.
Since all three hardware units employ pipelining, each cycle produces one completed result once the pipeline is filled.
Therefore, the throughput of all three modules is 1G.
Benefiting from its lower area and power consumption, the NLI Engine achieves 4.02× higher efficiency than NN-LUT and 4.29× higher efficiency than RI-LUT.
% \begin{wrapfigure}[16]{r}{8.0cm}
% \begin{adjustbox}{max width=\linewidth}
% \includegraphics[width=0.7\textwidth]{fig/figure_latency.pdf}
% \end{adjustbox}
% \caption{End-to-end latency comparison during the decoding stage among NLI, NN-LUT, and RI-LUT—three non-linear computing approaches—under identical area constraints with a sequence length of 256.} % 图片标题
% \label{fig:end2end}
% \end{wrapfigure}
% To further demonstrate that NLI can accelerate LLM inference, we designed a lightweight NPU that incorporates a linear compute unit and three types of non-linear compute units, and measured the per-token generation latency when running various LLMs.
% In this NPU, the same silicon area and the same clock frequency are allocated to implement each non-linear computing unit, and identical-scale systolic arrays (i.e., identical linear computation power) are employed. Consequently, speed-up in non-linear computation can be directly reflected in the end-to-end latency.
% As shown in Figure \ref{fig:end2end}, NLI reduces inference latency by roughly 10\% compared with the current SOTA across all models. This is mainly because each NLI computing unit has a small footprint, enabling higher parallelism per unit area than SOTA and thus shortening the time for nonlinear computation.
\vspace{-3mm}
\section{Conclusion}\label{sec:conclusion}
\vspace{-3mm}
In this paper, we propose \textbf{NLI}, a non-uniform linear interpolation method for efficiently approximating nonlinear functions in large language models. By formulating cutpoint selection as a dynamic programming problem, NLI achieves near-optimal accuracy with minimal hardware overhead. Experiments demonstrate that NLI maintains model accuracy without calibration, generalizes well across diverse models, and significantly reduces hardware resource usage compared to state-of-the-art methods. We believe NLI provides a practical solution for deploying large models efficiently on resource-limited hardware.
\section*{Ethics Statement}
This work does not involve human subjects, personal data, or sensitive content.
It focuses solely on algorithmic and hardware-level optimization of nonlinear function approximation.
Therefore, we believe it does not raise ethical concerns.
\section*{Reproducibility Statement}
We provide complete details of our algorithms, hyperparameters, and evaluation protocols in the main paper and appendix.
All models are evaluated on publicly available benchmarks (Wikitext-2, ARC, BoolQ, PIQA, HellaSwag, OBQA, LAMBADA, SIQA, WinoGrande, MMLU, HumanEval, GSM8k, and standard vision datasets).
The code for constructing lookup tables, performing DP cutpoint search, and reproducing our experiments will be released upon publication.
These resources will ensure full reproducibility of the reported results.
\bibliography{iclr2026_conference}
\bibliographystyle{iclr2026_conference}
\clearpage
\appendix
\section{Appendix}
\subsection{DP-Optimal Macro Cutpoint Search}
\begin{algorithm}[ht]
\caption{DP-Optimal Macro Cutpoint Search (NLI; mean relative error with endpoint clamping)}
\label{alg:nli-dp}
\KwIn{Sorted FP16 grid $\mathcal X=\{x_0,\dots,x_{N-1}\}$; target function $f(\cdot)$; number of \emph{cutpoints} $M\!\ge\!2$; small constant $\tau>0$.}
\KwOut{Optimal cutpoints $\mathcal B=\{b_0,\dots,b_{M-1}\}$ with $b_L\in\mathcal X$ and their values $\{f(b_L)\}$.}
\SetKwFunction{Err}{Err}
\SetKwFunction{Tail}{TailClamp}
\Step{1. Precompute values}{
\For{$k\leftarrow 0$ \KwTo $N-1$}{ $y_k \leftarrow f(x_k)$ }
}
\Step{2. Define error functionals}{
\tcp{Mean \emph{relative} error of the endpoint-anchored line on $[i,k]$}
\nl \(\displaystyle
\Err(i,k) \;\triangleq\; \frac{1}{k-i+1}\sum_{j=i}^{k}
\frac{\bigl|\,y_j - P_{i,k}(x_j)\,\bigr|}{\max\{|y_j|,\tau\}},
\quad P_{i,k}(x)=y_i+\frac{y_k-y_i}{x_k-x_i}\,(x-x_i)
\) \;
\tcp{Right-end clamping penalty for the last cutpoint (zero if $k=N{-}1$)}
\nl \(\displaystyle
\Tail(k) \;\triangleq\;
\begin{cases}
\frac{1}{N-1-k}\sum_{j=k+1}^{N-1}
\frac{\bigl|\,y_j - y_k\,\bigr|}{\max\{|y_j|,\tau\}}, & k<N-1,\\[7pt]
0, & k=N-1~,
\end{cases}
\)
}
\Step{3. Initialize DP tables}{
\tcp{$d\in\mathbb R^{M\times N}$ stores minimal prefix cost; $p\in\mathbb Z^{M\times N}$ stores predecessors}
Initialize $d[L,k]\leftarrow +\infty,\; p[L,k]\leftarrow -1$ for all $L,k$\;
\tcp{Left-end clamping: first cutpoint at $x_k$ uses constant $y_k$ on $[0,k]$}
\For{$k\leftarrow 0$ \KwTo $N-1$}{
$d[0,k]\leftarrow \frac{1}{k+1}\sum_{j=0}^{k}\frac{|\,y_j-y_k\,|}{\max\{|y_j|,\tau\}}$\;
$p[0,k]\leftarrow k$\;
}
}
\Step{4. Fill DP (macro endpoints)}{
\For{$L\leftarrow 1$ \KwTo $M-1$}{
\For{$k\leftarrow L$ \KwTo $N-1$}{
best $\leftarrow +\infty$;\quad arg $\leftarrow -1$\;
\For{$i\leftarrow L-1$ \KwTo $k-1$}{
val $\leftarrow d[L-1,i] + \Err(i,k) + \mathbf{1}_{\{L=M-1\}}\cdot\Tail(k)$\;
\If{val $<$ best}{ best $\leftarrow$ val;\; arg $\leftarrow i$ }
}
$d[L,k]\leftarrow$ best;\qquad $p[L,k]\leftarrow$ arg\;
}
}
}
\Step{5. Backtrack optimal cutpoints}{
$k^\star \leftarrow \arg\min_{k\in\{M-1,\dots,N-1\}} d[M-1,k]$\;
\tcp{Recover indices of $M$ cutpoints, from last to first}
$\text{idx}[M-1]\leftarrow k^\star$\;
\For{$L\leftarrow M-1$ \KwDownTo $1$}{
$\text{idx}[L-1]\leftarrow p[L,\,\text{idx}[L]]$\;
}
\For{$L\leftarrow 0$ \KwTo $M-1$}{
$b_L \leftarrow x_{\text{idx}[L]}$;\quad $f(b_L)\leftarrow y_{\text{idx}[L]}$\;
}
\Return{$\mathcal B=\{b_0,\dots,b_{M-1}\}$ and $\{f(b_L)\}_{L=0}^{M-1}$}\;
}
\end{algorithm}
\subsection{NLI Computation Flow}
\begin{algorithm}[ht]
\caption{NLI Computation Flow}
\label{alg:nli_forward}
\KwIn{FP16 input $x$; interval endpoints $Point[0{:}10]$ (11 points); per-interval scales $Mul[0{:}9]$ (10 values); LUT values $LUT[0{:}258]$ (259 points); uniform bins per macro-interval $D_n{=}32$ for intervals $1\ldots 8$.}
\KwOut{Nonlinear output $y$.}
\textbf{Constants:} $M{=}10$ intervals; indices are 0-based.\;
\BlankLine
\textbf{Preload registers:} $Point\_Reg \leftarrow Point$, $Mul\_Reg \leftarrow Mul$, $LUT\_Reg \leftarrow LUT$.\tcp*{one-time load}
\BlankLine
\textbf{Clamp input to domain:} $x \gets \mathrm{clip}\bigl(x,\,Point\_Reg[0],\,Point\_Reg[10]\bigr)$.\;
\BlankLine
\textbf{Locate interval} $Index \in \{0,\dots,9\}$:
\For{$i \gets 0$ \KwTo $9$}{
\If{$Point\_Reg[i] \le x < Point\_Reg[i{+}1]$}{ $Index \gets i$; \textbf{break}}
}
\tcp{Equivalently, this can be done via \texttt{bucketize}.}
\BlankLine
\textbf{Local coordinate in interval $Index$:}\\
$Temp \gets x - Point\_Reg[Index]$\tcp*{offset within the interval}
$Mul\_Temp \gets Temp \times Mul\_Reg[Index]$\tcp*{scaled position}
$Address \gets \lfloor Mul\_Temp \rfloor$\tcp*{integer bin (0 for intervals 0/9; 0..31 for 1..8)}
$Decimal \gets Mul\_Temp - Address$\tcp*{fractional part within the bin}
\BlankLine
\textbf{Global LUT index} $indices$:
\[
indices \gets
\begin{cases}
0 + Address, & \text{if } Index = 0,\\
1 + (Index{-}1)\cdot D_n + Address, & \text{if } Index \ge 1.
\end{cases}
\]
\BlankLine
\textbf{Linear interpolation from LUT:}\\
$Left \gets LUT\_Reg[indices]$,\quad $Right \gets LUT\_Reg[indices{+}1]$ \\
$y \gets Left + Decimal \times (Right - Left)$
\BlankLine
\textbf{Boundary saturation:}\\
\lIf{$x \le Point\_Reg[0]$}{$y \gets LUT\_Reg[0]$}
\lIf{$x \ge Point\_Reg[10]$}{$y \gets LUT\_Reg[258]$}
\Return{$y$}
\end{algorithm}
\subsection{LLM Usage Declaration}
We disclose that large language model (LLM) tools were used \emph{only} for language editing—copy-editing, stylistic smoothing, and minor rephrasing of prose—and limited formatting assistance. No LLM was involved in conceiving the research, proposing methods, generating technical content, writing code, running experiments, analyzing data, or drawing conclusions.
All substantive elements of this work were completed by the authors, including:
\begin{itemize}
\item problem formulation, research design, and overall narrative structure;
\item algorithmic development, theoretical reasoning/derivations, implementation, and debugging;
\item experimental design, data collection and preprocessing, execution, evaluation, and interpretation of results.
\end{itemize}
Any text suggestions produced by LLM tools were reviewed and edited by the authors. The authors accept full responsibility for the scientific validity, accuracy, and originality of the manuscript and affirm adherence to standards of academic integrity.
\subsection{Activation Coverage Measurement Protocol}
\label{app:act-measure}
\paragraph{Models and operators.}
We measure the pre-activation inputs to nonlinear operators (e.g., SiLU, RMSNorm, Softmax) across
the LLMs listed in Figure.~\ref{fig:range_and_compare} (a).
\paragraph{Data and setup.}
To align with our main evaluation, we use the same public corpora employed in Sec.~5 (e.g., Wikitext-2
for perplexity and the zero-shot suite for accuracy). All models are run in FP16 inference with the
same tokenization and maximum sequence length as in our evaluation.
\paragraph{Aggregation across layers and tokens.}
For each model, we collect activations from all layers over all tokens. We then compute the
0.05th and 99.95th percentiles of the resulting distribution. To form a symmetric interval, we set
$r_{\text{model}}=\max\{|\text{q}_{0.0005}|,\,|\text{q}_{0.9995}|\}$ and define the model-specific
range as $[-r_{\text{model}},\, r_{\text{model}}]$.
\paragraph{Cross-model coverage.}
We aggregate models by taking the union of their symmetric ranges, i.e., $r_{\max}=\max_{\text{model}} r_{\text{model}}$,
and define the LLM-typical domain as $[-r_{\max},\, r_{\max}]$. In our measurements, $r_{\max}\le 150$,
hence $[-150,150]$ \emph{covers $\ge$99.9\%} of observed activations.
\paragraph{Outliers.}
Values outside this domain account for $\le$0.1\% of activations and are clamped at runtime by our engine;
they have negligible impact on end-to-end accuracy.
\subsection{Full Experiment Results.}\label{appendx.fer}
\subsubsection{Supplementary Evaluation on Non-LLM Tasks}
\label{app:vision}
\begin{table}[ht]
\centering
\caption{Accuracy performance of NLI on vision models}
\begin{tabular}{p{2.4cm}cccr}
\toprule
\multirow{2}{*}{Model} & \multirow{2}{*}{Method} & mAP (\%) & Top-1 Acc (\%) \\[-1pt]
& & (obj det) & (cls) \\
\midrule
\multirow{2}{*}{DETR} & FP32 & 39.4 & — \\
& \textbf{NLI} & \textbf{39.4} & — \\[2pt]
\multirow{2}{*}{ViT-Small} & FP32 & — & 74.7 \\
& \textbf{NLI} & — & \textbf{74.7} \\[2pt]
\multirow{2}{*}{RT-DETR-L} & FP32 & 52.5 & — \\
& \textbf{NLI} & \textbf{52.5} & — \\[2pt]
\multirow{2}{*}{YOLOv8-M} & FP32 & 50.1 & — \\
& \textbf{NLI} & \textbf{50.1} & — \\
\bottomrule
\end{tabular}
\label{tab:nli_models}
\vspace{-2mm}
\end{table}
To further verify the generality of NLI beyond LLMs, we replaced the nonlinear
operators in representative vision models, including YOLOv8 \cite{varghese2024yolov8},
DETR \cite{carion2020detr}, ViT \cite{dosovitskiy2020vit}, and RT-DETR
\cite{zhao2024detrs}. As shown in Table~\ref{tab:nli_models}, employing NLI
for nonlinear operator computation does not incur any accuracy degradation,
demonstrating that the proposed framework is broadly applicable across
diverse architectures and modalities.
\begin{table}[ht]
\centering
\caption{NLI accuracy on multiple LLMs. Boldface rows highlight our method.}
\resizebox{\textwidth}{!}{
\begin{tabular}{p{2.5cm}ccccccccccccc}
\toprule
\multirow{2}{*}{Model} & \multirow{2}{*}{Method}
& \multicolumn{10}{c}{Zero-Shot (↑)}
& \multicolumn{1}{c}{Perplexity (↓)} \\
& & ARC-c & ARC-e & BoolQ & PIQA & HellaS & OBQA & Lam. & SIQA & WinoG. & Avg. & Wikitext-2 \\
\midrule
\multirow{2}{*}{Llama3-8B}
& FP32 & 53.16 & 77.78 & 81.25 & 80.74 & 79.10 & 44.80 & 75.66 & 47.08 & 73.40 & 68.11 & 6.14\\
& NN-LUT & 50.11 & 75.21 & 79.18 & 78.93 & 77.66 & 42.60 & 75.07 & 45.47 & 71.24 & 65.93 & 8.28\\
& \textbf{NLI}
& \textbf{53.67} & \textbf{77.78} & \textbf{81.41} & \textbf{80.74} & \textbf{79.23} & \textbf{44.80}
& \textbf{75.70} & \textbf{47.08} & \textbf{73.72} & \textbf{68.24} & \textbf{6.14}\\
\midrule
\multirow{2}{*}{Llama3-70B}
& FP32 & 64.25 & 86.03 & 85.29 & 84.44 & 85.00 & 48.60 & 79.35 & 50.67 & 80.43 & 73.78 & 2.86\\
& NN-LUT & 63.16 & 84.99 & 84.06 & 82.69 & 83.51 & 46.16 & 78.97 & 50.01 & 78.76 & 72.48 & 5.13\\
& \textbf{NLI}
& \textbf{64.33} & \textbf{86.07} & \textbf{85.08} & \textbf{84.39} & \textbf{84.92} & \textbf{49.00}
& \textbf{79.41} & \textbf{50.67} & \textbf{80.74} & \textbf{73.85} & \textbf{2.86}\\
\midrule
\multirow{2}{*}{Qwen2.5-7B}
& FP32 & 51.28 & 76.47 & 85.96 & 78.67 & 79.55 & 48.00 & 67.67 & 50.36 & 69.38 & 67.48 & 7.46\\
& NN-LUT & 19.62 & 25.38 & 37.83 & 52.12 & 25.61 & 28.20 & 0.00 & 33.01 & 49.41 & 30.13 & 28194\\
& \textbf{NLI}
& \textbf{51.37} & \textbf{76.39} & \textbf{85.96} & \textbf{79.82} & \textbf{79.46} & \textbf{48.20}
& \textbf{67.67} & \textbf{50.61} & \textbf{69.22} & \textbf{67.63} & \textbf{7.46}\\
\midrule
\multirow{2}{*}{Qwen2.5-32B}
& FP32 & 58.36 & 77.19 & 89.69 & 81.39 & 85.25 & 46.00 & 75.26 & 50.05 & 73.64 & 70.76 & 5.32\\
& NN-LUT & 20.99 & 23.95 & 38.63 & 53.92 & 26.64 & 28.80 & 0.00 & 33.21 & 50.12 & 30.70 & 70360\\
& \textbf{NLI}
& \textbf{58.28} & \textbf{77.44} & \textbf{89.72} & \textbf{81.23} & \textbf{85.18} & \textbf{45.40}
& \textbf{75.20} & \textbf{50.15} & \textbf{73.40} & \textbf{70.67} & \textbf{5.32}\\
\midrule
\multirow{2}{*}{Qwen1.5-110B}
& FP32 & 55.46 & 76.94 & 88.90 & 83.84 & 86.13 & 46.60 & 78.52 & 54.35 & 81.06 & 72.42 & 4.81\\
& NN-LUT & 52.01 & 74.08 & 83.87 & 80.32 & 83.77 & 42.40 & 75.99 & 51.32 & 77.96 & 69.08 & 6.83\\
& \textbf{NLI}
& \textbf{55.80} & \textbf{77.10} & \textbf{88.96} & \textbf{83.90} & \textbf{86.11} & \textbf{46.80}
& \textbf{78.36} & \textbf{54.25} & \textbf{81.06} & \textbf{72.48} & \textbf{4.81}\\
\midrule
\multirow{2}{*}{Qwen3-8B}
& FP32 & 55.29 & 80.93 & 86.70 & 77.69 & 74.92 & 41.40 & 64.04 & 51.84 & 67.32 & 66.68 & 9.72\\
& NN-LUT & 23.01 & 28.99 & 41.65 & 57.16 & 30.13 & 29.71 & 4.90 & 36.14 & 50.79 & 33.61 & 825.31\\
& \textbf{NLI}
& \textbf{55.29} & \textbf{80.93} & \textbf{86.70} & \textbf{77.90} & \textbf{75.11} & \textbf{41.53}
& \textbf{64.21} & \textbf{51.79} & \textbf{67.36} & \textbf{66.76} & \textbf{9.73}\\
\midrule
\multirow{2}{*}{Qwen3-30B-A3B}
& FP32 & 52.30 & 79.38 & 88.59 & 79.43 & 76.68 & 45.00 & 64.84 & 51.23 & 69.85 & 67.59 & 8.70\\
& NN-LUT & 50.71 & 77.86 & 87.03 & 77.69 & 75.23 & 44.13 & 60.00 & 49.76 & 68.01 & 65.60 & 10.76\\
& \textbf{NLI}
& \textbf{52.24} & \textbf{79.58} & \textbf{88.57} & \textbf{79.51} & \textbf{77.68} & \textbf{44.98}
& \textbf{64.91} & \textbf{51.46} & \textbf{69.94} & \textbf{67.65} & \textbf{8.70}\\
\bottomrule
\end{tabular}
}
\label{moret1}
\vspace{-3mm}
\end{table}
\subsubsection{Full Zero-Shot Evaluation on LLMs}
\label{app:full-zeroshot}
Table~\ref{moret1} reports the complete zero-shot evaluation results of NLI,
NN-LUT, and FP32 baselines on all considered LLMs. Compared with NN-LUT,
NLI consistently preserves baseline-level accuracy across tasks and scales,
while avoiding the severe degradation observed with NN-LUT.
\subsection{Additional NLI Visualisations and FP16 Cutpoints Tables}\label{appaddnul}
\begin{figure}[!htb]
\centering
\includegraphics[width=1.0\linewidth]{fig/appendix_1.pdf}
\caption{NLI approximation quality for eight representative nonlinear operators:
(a) \texttt{exp}, (b) \texttt{gelu}, (c) \texttt{rsqrt}, (d) \texttt{reciprocal},
(e) \texttt{hardswish}, (f) \texttt{mish}, (g) \texttt{sigmoid}, and (h) \texttt{tanh}.
For each operator, the left subplot shows the full-domain fit (top: FP32
reference vs.\ NLI; bottom: absolute error in log scale), while the right
subplot zooms into the high-curvature region. Red dots denote LUT cutpoints
generated under an \(2{+}8\times32{+}1\) budget. NLI closely overlaps the FP32
reference, keeping the worst-case absolute error within
\(1.5{\times}10^{-3}\) across the FP16 domain.}
\label{fig:more_nli}
\end{figure}
To demonstrate the breadth and stability of \textbf{NLI} across common nonlinear
operators, Figure~\ref{fig:more_nli} visualises fits for eight functions widely
used in modern models: \texttt{exp}, \texttt{gelu}, \texttt{rsqrt},
\texttt{reciprocal}, \texttt{hardswish}, \texttt{mish}, \texttt{sigmoid},
and \texttt{tanh}. Each panel contains two subplots: the left shows the full
FP16-domain behaviour (top: FP32 reference in teal and NLI in orange; bottom:
absolute error on a log scale), and the right provides a zoom-in around the
high-curvature region that typically dominates the global error budget.
Red dots mark the LUT cutpoints produced by our \(2{+}8\times32{+}1\) budget
(top-level endpoints plus uniformly spaced interior cuts). Across all eight
operators, NLI tracks the FP32 curve almost perfectly; the worst-case absolute
error remains within \(1.5{\times}10^{-3}\) over the entire domain and is
orders of magnitude smaller in most subranges.
Table~\ref{tab:fp16_cutpoints_appendix} lists the \emph{top-level} FP16
cutpoint endpoints (in decimal) that define 10 macro-intervals for each
operator. The full lookup table with \(2+8\times32+1=259\) entries is obtained
by placing 32 uniformly spaced cut points inside each macro-interval and adding
two boundary values. During inference, FP16 inputs that fall below the smallest
endpoint or above the largest endpoint are \emph{clamped} to the respective
boundary before lookup, which guarantees numerical stability for extreme values
(e.g., very small arguments in \texttt{rsqrt} and \texttt{reciprocal}).
For completeness, we also release the exact LUTs used in our experiments.
\begin{table}[t]
\centering
\resizebox{\textwidth}{!}{
\begin{tabular}{lccp{7.8cm}}
\toprule
Function & Range & \#Segments & Top-level cutpoints (FP16, decimal) \\ \midrule
gelu
& [$-5.5390625$, $65504.0$]
& $2{+}8\times32{+}1$
& \texttt{-5.5390625, -5.15625, -3.18359375, -0.98046875, -0.1229248046875, -0.00374603271484375, 0.0035247802734375, 0.11322021484375, 0.78076171875, 4.10546875, 65504.0} \\[3pt]
silu
& [$-20.359375$, $65504.0$]
& $2{+}8\times32{+}1$
& \texttt{-20.359375, -17.109375, -8.3671875, -1.9755859375, -0.255615234375, -0.007244110107421875, 0.0072174072265625, 0.228515625, 1.58203125, 10.46875, 65504.0} \\[3pt]
exp
& [$-17.34375$, $11.0859375$]
& $2{+}8\times32{+}1$
& \texttt{-17.34375, -15.171875, -8.890625, -5.2734375, -2.35546875, -0.3583984375, 0.91650390625, 3.451171875, 6.84765625, 10.9453125, 11.0859375} \\[3pt]
reciprocal
& [$1.5318394{\times}10^{-5}$, $65504.0$]
& $2{+}8\times32{+}1$
& \texttt{1.5318394e-05, 2.2590160e-05, 4.6992302e-04, 7.0533752e-03, 8.8378906e-02, 1.07421875, 15.546875, 244.5, 3694.0, 46560.0, 65504.0} \\[3pt]
rsqrt
& [$5.9604645{\times}10^{-8}$, $65504.0$]
& $2{+}8\times32{+}1$
& \texttt{5.9604645e-08, 7.7486038e-07, 1.1140108e-04, 1.8644333e-03, 3.0029297e-02, 0.48193359375, 7.7734375, 129.75, 2406.0, 47456.0, 65504.0} \\[3pt]
hardswish
& [$-3.0$, $65504.0$]
& $2{+}8\times32{+}1$
& \texttt{-3.0, -2.984375, -1.87890625, -0.5390625, -0.059326171875, -0.000743865966796875, 0.0034942626953125, 0.11968994140625, 0.78369140625, 3.001953125, 65504.0} \\[3pt]
tanh
& [$-4.5078125$, $4.5078125$]
& $2{+}8\times32{+}1$
& \texttt{-4.5078125, -3.79296875, -1.55078125, -0.5302734375, -0.028564453125, 0.0364990234375, 0.423828125, 1.076171875, 2.0390625, 4.0625, 4.5078125} \\[3pt]
mish
& [$-20.34375$, $65504.0$]
& $2{+}8\times32{+}1$
& \texttt{-20.34375, -19.90625, -10.921875, -6.2265625, -1.615234375, -0.237060546875, -0.00699615478515625, 0.01538848876953125, 0.491455078125, 4.70703125, 65504.0} \\[3pt]
sigmoid
& [$-17.34375$, $8.3203125$]
& $2{+}8\times32{+}1$
& \texttt{-17.34375, -15.765625, -10.65625, -8.15625, -6.3046875, -4.421875, -2.6640625, -0.7998046875, 1.9462890625, 6.90234375, 8.3203125} \\
\bottomrule
\end{tabular}
}
\caption{Top-level cutpoint endpoints used by NLI. Each row lists 11 FP16 cutpoints (decimal) that define 10 macro-intervals; placing 32 uniformly spaced cut points inside each macro-interval plus two boundary values yields a total of \(2{+}8\times32{+} 1=259\) entries. \emph{Inference uses clamping: FP16-domain inputs below the smallest cutpoint or above the largest cutpoint are clipped to the respective endpoint.}}
\label{tab:fp16_cutpoints_appendix}
\end{table}
\end{document}
%%%%% NEW MATH DEFINITIONS %%%%%
\usepackage{amsmath,amsfonts,bm}
% Mark sections of captions for referring to divisions of figures
\newcommand{\figleft}{{\em (Left)}}
\newcommand{\figcenter}{{\em (Center)}}
\newcommand{\figright}{{\em (Right)}}
\newcommand{\figtop}{{\em (Top)}}
\newcommand{\figbottom}{{\em (Bottom)}}
\newcommand{\captiona}{{\em (a)}}
\newcommand{\captionb}{{\em (b)}}
\newcommand{\captionc}{{\em (c)}}
\newcommand{\captiond}{{\em (d)}}
% Highlight a newly defined term
\newcommand{\newterm}[1]{{\bf #1}}
% Figure reference, lower-case.
\def\figref#1{figure~\ref{#1}}
% Figure reference, capital. For start of sentence
\def\Figref#1{Figure~\ref{#1}}
\def\twofigref#1#2{figures \ref{#1} and \ref{#2}}
\def\quadfigref#1#2#3#4{figures \ref{#1}, \ref{#2}, \ref{#3} and \ref{#4}}
% Section reference, lower-case.
\def\secref#1{section~\ref{#1}}
% Section reference, capital.
\def\Secref#1{Section~\ref{#1}}
% Reference to two sections.
\def\twosecrefs#1#2{sections \ref{#1} and \ref{#2}}
% Reference to three sections.
\def\secrefs#1#2#3{sections \ref{#1}, \ref{#2} and \ref{#3}}
% Reference to an equation, lower-case.
\def\eqref#1{equation~\ref{#1}}
% Reference to an equation, upper case
\def\Eqref#1{Equation~\ref{#1}}
% A raw reference to an equation---avoid using if possible
\def\plaineqref#1{\ref{#1}}
% Reference to a chapter, lower-case.
\def\chapref#1{chapter~\ref{#1}}
% Reference to an equation, upper case.
\def\Chapref#1{Chapter~\ref{#1}}
% Reference to a range of chapters
\def\rangechapref#1#2{chapters\ref{#1}--\ref{#2}}
% Reference to an algorithm, lower-case.
\def\algref#1{algorithm~\ref{#1}}
% Reference to an algorithm, upper case.
\def\Algref#1{Algorithm~\ref{#1}}
\def\twoalgref#1#2{algorithms \ref{#1} and \ref{#2}}
\def\Twoalgref#1#2{Algorithms \ref{#1} and \ref{#2}}
% Reference to a part, lower case
\def\partref#1{part~\ref{#1}}
% Reference to a part, upper case
\def\Partref#1{Part~\ref{#1}}
\def\twopartref#1#2{parts \ref{#1} and \ref{#2}}
\def\ceil#1{\lceil #1 \rceil}
\def\floor#1{\lfloor #1 \rfloor}
\def\1{\bm{1}}
\newcommand{\train}{\mathcal{D}}
\newcommand{\valid}{\mathcal{D_{\mathrm{valid}}}}
\newcommand{\test}{\mathcal{D_{\mathrm{test}}}}
\def\eps{{\epsilon}}
% Random variables
\def\reta{{\textnormal{$\eta$}}}
\def\ra{{\textnormal{a}}}
\def\rb{{\textnormal{b}}}
\def\rc{{\textnormal{c}}}
\def\rd{{\textnormal{d}}}
\def\re{{\textnormal{e}}}
\def\rf{{\textnormal{f}}}
\def\rg{{\textnormal{g}}}
\def\rh{{\textnormal{h}}}
\def\ri{{\textnormal{i}}}
\def\rj{{\textnormal{j}}}
\def\rk{{\textnormal{k}}}
\def\rl{{\textnormal{l}}}
% rm is already a command, just don't name any random variables m
\def\rn{{\textnormal{n}}}
\def\ro{{\textnormal{o}}}
\def\rp{{\textnormal{p}}}
\def\rq{{\textnormal{q}}}
\def\rr{{\textnormal{r}}}
\def\rs{{\textnormal{s}}}
\def\rt{{\textnormal{t}}}
\def\ru{{\textnormal{u}}}
\def\rv{{\textnormal{v}}}
\def\rw{{\textnormal{w}}}
\def\rx{{\textnormal{x}}}
\def\ry{{\textnormal{y}}}
\def\rz{{\textnormal{z}}}
% Random vectors
\def\rvepsilon{{\mathbf{\epsilon}}}
\def\rvtheta{{\mathbf{\theta}}}
\def\rva{{\mathbf{a}}}
\def\rvb{{\mathbf{b}}}
\def\rvc{{\mathbf{c}}}
\def\rvd{{\mathbf{d}}}
\def\rve{{\mathbf{e}}}
\def\rvf{{\mathbf{f}}}
\def\rvg{{\mathbf{g}}}
\def\rvh{{\mathbf{h}}}
\def\rvu{{\mathbf{i}}}
\def\rvj{{\mathbf{j}}}
\def\rvk{{\mathbf{k}}}
\def\rvl{{\mathbf{l}}}
\def\rvm{{\mathbf{m}}}
\def\rvn{{\mathbf{n}}}
\def\rvo{{\mathbf{o}}}
\def\rvp{{\mathbf{p}}}
\def\rvq{{\mathbf{q}}}
\def\rvr{{\mathbf{r}}}
\def\rvs{{\mathbf{s}}}
\def\rvt{{\mathbf{t}}}
\def\rvu{{\mathbf{u}}}
\def\rvv{{\mathbf{v}}}
\def\rvw{{\mathbf{w}}}
\def\rvx{{\mathbf{x}}}
\def\rvy{{\mathbf{y}}}
\def\rvz{{\mathbf{z}}}
% Elements of random vectors
\def\erva{{\textnormal{a}}}
\def\ervb{{\textnormal{b}}}
\def\ervc{{\textnormal{c}}}
\def\ervd{{\textnormal{d}}}
\def\erve{{\textnormal{e}}}
\def\ervf{{\textnormal{f}}}
\def\ervg{{\textnormal{g}}}
\def\ervh{{\textnormal{h}}}
\def\ervi{{\textnormal{i}}}
\def\ervj{{\textnormal{j}}}
\def\ervk{{\textnormal{k}}}
\def\ervl{{\textnormal{l}}}
\def\ervm{{\textnormal{m}}}
\def\ervn{{\textnormal{n}}}
\def\ervo{{\textnormal{o}}}
\def\ervp{{\textnormal{p}}}
\def\ervq{{\textnormal{q}}}
\def\ervr{{\textnormal{r}}}
\def\ervs{{\textnormal{s}}}
\def\ervt{{\textnormal{t}}}
\def\ervu{{\textnormal{u}}}
\def\ervv{{\textnormal{v}}}
\def\ervw{{\textnormal{w}}}
\def\ervx{{\textnormal{x}}}
\def\ervy{{\textnormal{y}}}
\def\ervz{{\textnormal{z}}}
% Random matrices
\def\rmA{{\mathbf{A}}}
\def\rmB{{\mathbf{B}}}
\def\rmC{{\mathbf{C}}}
\def\rmD{{\mathbf{D}}}
\def\rmE{{\mathbf{E}}}
\def\rmF{{\mathbf{F}}}
\def\rmG{{\mathbf{G}}}
\def\rmH{{\mathbf{H}}}
\def\rmI{{\mathbf{I}}}
\def\rmJ{{\mathbf{J}}}
\def\rmK{{\mathbf{K}}}
\def\rmL{{\mathbf{L}}}
\def\rmM{{\mathbf{M}}}
\def\rmN{{\mathbf{N}}}
\def\rmO{{\mathbf{O}}}
\def\rmP{{\mathbf{P}}}
\def\rmQ{{\mathbf{Q}}}
\def\rmR{{\mathbf{R}}}
\def\rmS{{\mathbf{S}}}
\def\rmT{{\mathbf{T}}}
\def\rmU{{\mathbf{U}}}
\def\rmV{{\mathbf{V}}}
\def\rmW{{\mathbf{W}}}
\def\rmX{{\mathbf{X}}}
\def\rmY{{\mathbf{Y}}}
\def\rmZ{{\mathbf{Z}}}
% Elements of random matrices
\def\ermA{{\textnormal{A}}}
\def\ermB{{\textnormal{B}}}
\def\ermC{{\textnormal{C}}}
\def\ermD{{\textnormal{D}}}
\def\ermE{{\textnormal{E}}}
\def\ermF{{\textnormal{F}}}
\def\ermG{{\textnormal{G}}}
\def\ermH{{\textnormal{H}}}
\def\ermI{{\textnormal{I}}}
\def\ermJ{{\textnormal{J}}}
\def\ermK{{\textnormal{K}}}
\def\ermL{{\textnormal{L}}}
\def\ermM{{\textnormal{M}}}
\def\ermN{{\textnormal{N}}}
\def\ermO{{\textnormal{O}}}
\def\ermP{{\textnormal{P}}}
\def\ermQ{{\textnormal{Q}}}
\def\ermR{{\textnormal{R}}}
\def\ermS{{\textnormal{S}}}
\def\ermT{{\textnormal{T}}}
\def\ermU{{\textnormal{U}}}
\def\ermV{{\textnormal{V}}}
\def\ermW{{\textnormal{W}}}
\def\ermX{{\textnormal{X}}}
\def\ermY{{\textnormal{Y}}}
\def\ermZ{{\textnormal{Z}}}
% Vectors
\def\vzero{{\bm{0}}}
\def\vone{{\bm{1}}}
\def\vmu{{\bm{\mu}}}
\def\vtheta{{\bm{\theta}}}
\def\va{{\bm{a}}}
\def\vb{{\bm{b}}}
\def\vc{{\bm{c}}}
\def\vd{{\bm{d}}}
\def\ve{{\bm{e}}}
\def\vf{{\bm{f}}}
\def\vg{{\bm{g}}}
\def\vh{{\bm{h}}}
\def\vi{{\bm{i}}}
\def\vj{{\bm{j}}}
\def\vk{{\bm{k}}}
\def\vl{{\bm{l}}}
\def\vm{{\bm{m}}}
\def\vn{{\bm{n}}}
\def\vo{{\bm{o}}}
\def\vp{{\bm{p}}}
\def\vq{{\bm{q}}}
\def\vr{{\bm{r}}}
\def\vs{{\bm{s}}}
\def\vt{{\bm{t}}}
\def\vu{{\bm{u}}}
\def\vv{{\bm{v}}}
\def\vw{{\bm{w}}}
\def\vx{{\bm{x}}}
\def\vy{{\bm{y}}}
\def\vz{{\bm{z}}}
% Elements of vectors
\def\evalpha{{\alpha}}
\def\evbeta{{\beta}}
\def\evepsilon{{\epsilon}}
\def\evlambda{{\lambda}}
\def\evomega{{\omega}}
\def\evmu{{\mu}}
\def\evpsi{{\psi}}
\def\evsigma{{\sigma}}
\def\evtheta{{\theta}}
\def\eva{{a}}
\def\evb{{b}}
\def\evc{{c}}
\def\evd{{d}}
\def\eve{{e}}
\def\evf{{f}}
\def\evg{{g}}
\def\evh{{h}}
\def\evi{{i}}
\def\evj{{j}}
\def\evk{{k}}
\def\evl{{l}}
\def\evm{{m}}
\def\evn{{n}}
\def\evo{{o}}
\def\evp{{p}}
\def\evq{{q}}
\def\evr{{r}}
\def\evs{{s}}
\def\evt{{t}}
\def\evu{{u}}
\def\evv{{v}}
\def\evw{{w}}
\def\evx{{x}}
\def\evy{{y}}
\def\evz{{z}}
% Matrix
\def\mA{{\bm{A}}}
\def\mB{{\bm{B}}}
\def\mC{{\bm{C}}}
\def\mD{{\bm{D}}}
\def\mE{{\bm{E}}}
\def\mF{{\bm{F}}}
\def\mG{{\bm{G}}}
\def\mH{{\bm{H}}}
\def\mI{{\bm{I}}}
\def\mJ{{\bm{J}}}
\def\mK{{\bm{K}}}
\def\mL{{\bm{L}}}
\def\mM{{\bm{M}}}
\def\mN{{\bm{N}}}
\def\mO{{\bm{O}}}
\def\mP{{\bm{P}}}
\def\mQ{{\bm{Q}}}
\def\mR{{\bm{R}}}
\def\mS{{\bm{S}}}
\def\mT{{\bm{T}}}
\def\mU{{\bm{U}}}
\def\mV{{\bm{V}}}
\def\mW{{\bm{W}}}
\def\mX{{\bm{X}}}
\def\mY{{\bm{Y}}}
\def\mZ{{\bm{Z}}}
\def\mBeta{{\bm{\beta}}}
\def\mPhi{{\bm{\Phi}}}
\def\mLambda{{\bm{\Lambda}}}
\def\mSigma{{\bm{\Sigma}}}
% Tensor
\DeclareMathAlphabet{\mathsfit}{\encodingdefault}{\sfdefault}{m}{sl}
\SetMathAlphabet{\mathsfit}{bold}{\encodingdefault}{\sfdefault}{bx}{n}
\newcommand{\tens}[1]{\bm{\mathsfit{#1}}}
\def\tA{{\tens{A}}}
\def\tB{{\tens{B}}}
\def\tC{{\tens{C}}}
\def\tD{{\tens{D}}}
\def\tE{{\tens{E}}}
\def\tF{{\tens{F}}}
\def\tG{{\tens{G}}}
\def\tH{{\tens{H}}}
\def\tI{{\tens{I}}}
\def\tJ{{\tens{J}}}
\def\tK{{\tens{K}}}
\def\tL{{\tens{L}}}
\def\tM{{\tens{M}}}
\def\tN{{\tens{N}}}
\def\tO{{\tens{O}}}
\def\tP{{\tens{P}}}
\def\tQ{{\tens{Q}}}
\def\tR{{\tens{R}}}
\def\tS{{\tens{S}}}
\def\tT{{\tens{T}}}
\def\tU{{\tens{U}}}
\def\tV{{\tens{V}}}
\def\tW{{\tens{W}}}
\def\tX{{\tens{X}}}
\def\tY{{\tens{Y}}}
\def\tZ{{\tens{Z}}}
% Graph
\def\gA{{\mathcal{A}}}
\def\gB{{\mathcal{B}}}
\def\gC{{\mathcal{C}}}
\def\gD{{\mathcal{D}}}
\def\gE{{\mathcal{E}}}
\def\gF{{\mathcal{F}}}
\def\gG{{\mathcal{G}}}
\def\gH{{\mathcal{H}}}
\def\gI{{\mathcal{I}}}
\def\gJ{{\mathcal{J}}}
\def\gK{{\mathcal{K}}}
\def\gL{{\mathcal{L}}}
\def\gM{{\mathcal{M}}}
\def\gN{{\mathcal{N}}}
\def\gO{{\mathcal{O}}}
\def\gP{{\mathcal{P}}}
\def\gQ{{\mathcal{Q}}}
\def\gR{{\mathcal{R}}}
\def\gS{{\mathcal{S}}}
\def\gT{{\mathcal{T}}}
\def\gU{{\mathcal{U}}}
\def\gV{{\mathcal{V}}}
\def\gW{{\mathcal{W}}}
\def\gX{{\mathcal{X}}}
\def\gY{{\mathcal{Y}}}
\def\gZ{{\mathcal{Z}}}
% Sets
\def\sA{{\mathbb{A}}}
\def\sB{{\mathbb{B}}}
\def\sC{{\mathbb{C}}}
\def\sD{{\mathbb{D}}}
% Don't use a set called E, because this would be the same as our symbol
% for expectation.
\def\sF{{\mathbb{F}}}
\def\sG{{\mathbb{G}}}
\def\sH{{\mathbb{H}}}
\def\sI{{\mathbb{I}}}
\def\sJ{{\mathbb{J}}}
\def\sK{{\mathbb{K}}}
\def\sL{{\mathbb{L}}}
\def\sM{{\mathbb{M}}}
\def\sN{{\mathbb{N}}}
\def\sO{{\mathbb{O}}}
\def\sP{{\mathbb{P}}}
\def\sQ{{\mathbb{Q}}}
\def\sR{{\mathbb{R}}}
\def\sS{{\mathbb{S}}}
\def\sT{{\mathbb{T}}}
\def\sU{{\mathbb{U}}}
\def\sV{{\mathbb{V}}}
\def\sW{{\mathbb{W}}}
\def\sX{{\mathbb{X}}}
\def\sY{{\mathbb{Y}}}
\def\sZ{{\mathbb{Z}}}
% Entries of a matrix
\def\emLambda{{\Lambda}}
\def\emA{{A}}
\def\emB{{B}}
\def\emC{{C}}
\def\emD{{D}}
\def\emE{{E}}
\def\emF{{F}}
\def\emG{{G}}
\def\emH{{H}}
\def\emI{{I}}
\def\emJ{{J}}
\def\emK{{K}}
\def\emL{{L}}
\def\emM{{M}}
\def\emN{{N}}
\def\emO{{O}}
\def\emP{{P}}
\def\emQ{{Q}}
\def\emR{{R}}
\def\emS{{S}}
\def\emT{{T}}
\def\emU{{U}}
\def\emV{{V}}
\def\emW{{W}}
\def\emX{{X}}
\def\emY{{Y}}
\def\emZ{{Z}}
\def\emSigma{{\Sigma}}
% entries of a tensor
% Same font as tensor, without \bm wrapper
\newcommand{\etens}[1]{\mathsfit{#1}}
\def\etLambda{{\etens{\Lambda}}}
\def\etA{{\etens{A}}}
\def\etB{{\etens{B}}}
\def\etC{{\etens{C}}}
\def\etD{{\etens{D}}}
\def\etE{{\etens{E}}}
\def\etF{{\etens{F}}}
\def\etG{{\etens{G}}}
\def\etH{{\etens{H}}}
\def\etI{{\etens{I}}}
\def\etJ{{\etens{J}}}
\def\etK{{\etens{K}}}
\def\etL{{\etens{L}}}
\def\etM{{\etens{M}}}
\def\etN{{\etens{N}}}
\def\etO{{\etens{O}}}
\def\etP{{\etens{P}}}
\def\etQ{{\etens{Q}}}
\def\etR{{\etens{R}}}
\def\etS{{\etens{S}}}
\def\etT{{\etens{T}}}
\def\etU{{\etens{U}}}
\def\etV{{\etens{V}}}
\def\etW{{\etens{W}}}
\def\etX{{\etens{X}}}
\def\etY{{\etens{Y}}}
\def\etZ{{\etens{Z}}}
% The true underlying data generating distribution
\newcommand{\pdata}{p_{\rm{data}}}
% The empirical distribution defined by the training set
\newcommand{\ptrain}{\hat{p}_{\rm{data}}}
\newcommand{\Ptrain}{\hat{P}_{\rm{data}}}
% The model distribution
\newcommand{\pmodel}{p_{\rm{model}}}
\newcommand{\Pmodel}{P_{\rm{model}}}
\newcommand{\ptildemodel}{\tilde{p}_{\rm{model}}}
% Stochastic autoencoder distributions
\newcommand{\pencode}{p_{\rm{encoder}}}
\newcommand{\pdecode}{p_{\rm{decoder}}}
\newcommand{\precons}{p_{\rm{reconstruct}}}
\newcommand{\laplace}{\mathrm{Laplace}} % Laplace distribution
\newcommand{\E}{\mathbb{E}}
\newcommand{\Ls}{\mathcal{L}}
\newcommand{\R}{\mathbb{R}}
\newcommand{\emp}{\tilde{p}}
\newcommand{\lr}{\alpha}
\newcommand{\reg}{\lambda}
\newcommand{\rect}{\mathrm{rectifier}}
\newcommand{\softmax}{\mathrm{softmax}}
\newcommand{\sigmoid}{\sigma}
\newcommand{\softplus}{\zeta}
\newcommand{\KL}{D_{\mathrm{KL}}}
\newcommand{\Var}{\mathrm{Var}}
\newcommand{\standarderror}{\mathrm{SE}}
\newcommand{\Cov}{\mathrm{Cov}}
% Wolfram Mathworld says $L^2$ is for function spaces and $\ell^2$ is for vectors
% But then they seem to use $L^2$ for vectors throughout the site, and so does
% wikipedia.
\newcommand{\normlzero}{L^0}
\newcommand{\normlone}{L^1}
\newcommand{\normltwo}{L^2}
\newcommand{\normlp}{L^p}
\newcommand{\normmax}{L^\infty}
\newcommand{\parents}{Pa} % See usage in notation.tex. Chosen to match Daphne's book.
\DeclareMathOperator*{\argmax}{arg\,max}
\DeclareMathOperator*{\argmin}{arg\,min}
\DeclareMathOperator{\sign}{sign}
\DeclareMathOperator{\Tr}{Tr}
\let\ab\allowbreak
\begin{thebibliography}{42}
\providecommand{\natexlab}[1]{#1}
\providecommand{\url}[1]{\texttt{#1}}
\expandafter\ifx\csname urlstyle\endcsname\relax
\providecommand{\doi}[1]{doi: #1}\else
\providecommand{\doi}{doi: \begingroup \urlstyle{rm}\Url}\fi
\bibitem[Bachrach et~al.(2012)Bachrach, Vo, Richards, Lee, Waterman, Avi{\v{z}}ienis, Wawrzynek, and Asanovi{\'c}]{bachrach2012chisel}
Jonathan Bachrach, Huy Vo, Brian Richards, Yunsup Lee, Andrew Waterman, Rimas Avi{\v{z}}ienis, John Wawrzynek, and Krste Asanovi{\'c}.
\newblock Chisel: constructing hardware in a scala embedded language.
\newblock In \emph{Proceedings of the 49th annual design automation conference}, pp.\ 1216--1225, 2012.
\bibitem[Bai et~al.(2023)Bai, Bai, Chu, Cui, Dang, Deng, Fan, Ge, Han, Huang, et~al.]{bai2023qwen}
Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu~Han, Fei Huang, et~al.
\newblock Qwen technical report.
\newblock \emph{arXiv preprint arXiv:2309.16609}, 2023.
\bibitem[Bisk et~al.(2020)Bisk, Zellers, Gao, Choi, et~al.]{bisk2020piqa}
Yonatan Bisk, Rowan Zellers, Jianfeng Gao, Yejin Choi, et~al.
\newblock Piqa: Reasoning about physical commonsense in natural language.
\newblock In \emph{Proceedings of the AAAI conference on artificial intelligence}, volume~34, pp.\ 7432--7439, 2020.
\bibitem[Cantoni(1971)]{cantoni1971optimal}
Antonio Cantoni.
\newblock Optimal curve fitting with piecewise linear functions.
\newblock \emph{IEEE Transactions on Computers}, 20\penalty0 (01):\penalty0 59--67, 1971.
\bibitem[Carion et~al.(2020)Carion, Massa, Synnaeve, Usunier, Kirillov, and Zagoruyko]{carion2020detr}
Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexander Kirillov, and Sergey Zagoruyko.
\newblock End-to-end object detection with transformers.
\newblock In \emph{European conference on computer vision}, pp.\ 213--229. Springer, 2020.
\bibitem[Chen et~al.(2021)Chen, Tworek, Jun, Yuan, Pinto, Kaplan, Edwards, Burda, Joseph, Brockman, et~al.]{chen2021evaluating}
Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde De~Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, et~al.
\newblock Evaluating large language models trained on code.
\newblock \emph{arXiv preprint arXiv:2107.03374}, 2021.
\bibitem[Clark et~al.(2019)Clark, Lee, Chang, Kwiatkowski, Collins, and Toutanova]{clark2019boolq}
Christopher Clark, Kenton Lee, Ming-Wei Chang, Tom Kwiatkowski, Michael Collins, and Kristina Toutanova.
\newblock Boolq: Exploring the surprising difficulty of natural yes/no questions.
\newblock \emph{arXiv preprint arXiv:1905.10044}, 2019.
\bibitem[Clark et~al.(2018)Clark, Cowhey, Etzioni, Khot, Sabharwal, Schoenick, and Tafjord]{clark2018think}
Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord.
\newblock Think you have solved question answering? try arc, the ai2 reasoning challenge.
\newblock \emph{arXiv preprint arXiv:1803.05457}, 2018.
\bibitem[Cobbe et~al.(2021)Cobbe, Kosaraju, Bavarian, Chen, Jun, Kaiser, Plappert, Tworek, Hilton, Nakano, et~al.]{cobbe2021training}
Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, et~al.
\newblock Training verifiers to solve math word problems.
\newblock \emph{arXiv preprint arXiv:2110.14168}, 2021.
\bibitem[Dosovitskiy et~al.(2020)Dosovitskiy, Beyer, Kolesnikov, Weissenborn, Zhai, Unterthiner, Dehghani, Minderer, Heigold, Gelly, et~al.]{dosovitskiy2020vit}
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et~al.
\newblock An image is worth 16x16 words: Transformers for image recognition at scale.
\newblock \emph{arXiv preprint arXiv:2010.11929}, 2020.
\bibitem[Genc et~al.(2021)Genc, Kim, Amid, Haj-Ali, Iyer, Prakash, Zhao, Grubb, Liew, Mao, et~al.]{genc2021gemmini}
Hasan Genc, Seah Kim, Alon Amid, Ameer Haj-Ali, Vighnesh Iyer, Pranav Prakash, Jerry Zhao, Daniel Grubb, Harrison Liew, Howard Mao, et~al.
\newblock Gemmini: Enabling systematic deep-learning architecture evaluation via full-stack integration.
\newblock In \emph{2021 58th ACM/IEEE Design Automation Conference (DAC)}, pp.\ 769--774. IEEE, 2021.
\bibitem[Google(2016)]{tpu_transformation}
Google.
\newblock Tpu transformation: A look back at 10 years of our ai-specialized chips, 2016.
\newblock URL \url{https://cloud.google.com/transform/ai-specialized-chips-tpu-history-gen-ai}.
\newblock Accessed on March 24, 2025.
\bibitem[Guo et~al.(2024)Guo, Zhu, Yang, Xie, Dong, Zhang, Chen, Bi, Wu, Li, et~al.]{guo2024deepseek}
Daya Guo, Qihao Zhu, Dejian Yang, Zhenda Xie, Kai Dong, Wentao Zhang, Guanting Chen, Xiao Bi, Yu~Wu, YK~Li, et~al.
\newblock Deepseek-coder: When the large language model meets programming--the rise of code intelligence.
\newblock \emph{arXiv preprint arXiv:2401.14196}, 2024.
\bibitem[Hendrycks et~al.(2020)Hendrycks, Burns, Basart, Zou, Mazeika, Song, and Steinhardt]{hendrycks2020measuring}
Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt.
\newblock Measuring massive multitask language understanding.
\newblock \emph{arXiv preprint arXiv:2009.03300}, 2020.
\bibitem[Hu et~al.(2025)Hu, Cheng, Yang, Xu, Yuan, Yu, Xu, Jiang, and Zhou]{hu2025ostquant}
Xing Hu, Yuan Cheng, Dawei Yang, Zukang Xu, Zhihang Yuan, Jiangyong Yu, Chen Xu, Zhe Jiang, and Sifan Zhou.
\newblock Ostquant: Refining large language model quantization with orthogonal and scaling transformations for better distribution fitting.
\newblock \emph{arXiv preprint arXiv:2501.13987}, 2025.
\bibitem[Karst(1958)]{karst1958linear}
Otto~J Karst.
\newblock Linear curve fitting using least deviations.
\newblock \emph{Journal of the American Statistical Association}, 53\penalty0 (281):\penalty0 118--132, 1958.
\bibitem[Kim et~al.(2023)Kim, Lee, Choi, Han, and Lee]{kim2023range}
Janghyeon Kim, Janghwan Lee, Jungwook Choi, JeongHo Han, and Sangheon Lee.
\newblock Range-invariant approximation of non-linear operations for efficient bert fine-tuning.
\newblock In \emph{2023 60th ACM/IEEE Design Automation Conference (DAC)}, pp.\ 1--6. IEEE, 2023.
\bibitem[Kim et~al.(2021)Kim, Gholami, Yao, Mahoney, and Keutzer]{kim2021bert}
Sehoon Kim, Amir Gholami, Zhewei Yao, Michael~W Mahoney, and Kurt Keutzer.
\newblock I-bert: Integer-only bert quantization.
\newblock In \emph{International conference on machine learning}, pp.\ 5506--5518. PMLR, 2021.
\bibitem[Li et~al.(2024)Li, Tang, Zhao, Nie, and Wen]{generation}
Junyi Li, Tianyi Tang, Wayne~Xin Zhao, Jian-Yun Nie, and Ji-Rong Wen.
\newblock Pre-trained language models for text generation: A survey.
\newblock \emph{ACM Computing Surveys}, 56\penalty0 (9):\penalty0 1--39, 2024.
\bibitem[Lin et~al.(2024)Lin, Tang, Tang, Yang, Chen, Wang, Xiao, Dang, Gan, and Han]{lin2024awq}
Ji~Lin, Jiaming Tang, Haotian Tang, Shang Yang, Wei-Ming Chen, Wei-Chen Wang, Guangxuan Xiao, Xingyu Dang, Chuang Gan, and Song Han.
\newblock Awq: Activation-aware weight quantization for on-device llm compression and acceleration.
\newblock \emph{Proceedings of Machine Learning and Systems}, 6:\penalty0 87--100, 2024.
\bibitem[Liu et~al.(2015)Liu, Anh, McGree, Kozan, and Wolff]{high}
Shen Liu, Vo~Anh, James McGree, Erhan Kozan, and Rodney~C Wolff.
\newblock A new approach to spatial data interpolation using higher-order statistics.
\newblock \emph{Stochastic Environmental Research and Risk Assessment}, 29:\penalty0 1679--1690, 2015.
\bibitem[Merity et~al.(2016)Merity, Xiong, Bradbury, and Socher]{merity2016wiki}
Stephen Merity, Caiming Xiong, James Bradbury, and Richard Socher.
\newblock Pointer sentinel mixture models.
\newblock \emph{arXiv preprint arXiv:1609.07843}, 2016.
\bibitem[Mihaylov et~al.(2018)Mihaylov, Clark, Khot, and Sabharwal]{mihaylov2018obqa}
Todor Mihaylov, Peter Clark, Tushar Khot, and Ashish Sabharwal.
\newblock Can a suit of armor conduct electricity? a new dataset for open book question answering.
\newblock \emph{arXiv preprint arXiv:1809.02789}, 2018.
\bibitem[Muchnick(1997)]{muchnick1997dc}
Steven Muchnick.
\newblock \emph{Advanced compiler design implementation}.
\newblock Morgan kaufmann, 1997.
\bibitem[Naeem et~al.(2023)Naeem, Khan, Xian, Afzal, Stricker, Van~Gool, and Tombari]{naeem2023i2mvformer}
Muhammad~Ferjad Naeem, Muhammad Gul Zain~Ali Khan, Yongqin Xian, Muhammad~Zeshan Afzal, Didier Stricker, Luc Van~Gool, and Federico Tombari.
\newblock I2mvformer: Large language model generated multi-view document supervision for zero-shot image classification.
\newblock In \emph{Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition}, pp.\ 15169--15179, 2023.
\bibitem[NVDLA(2017)]{nvdla_primer}
NVDLA.
\newblock Nvdla primer — nvdla documentation, 2017.
\newblock URL \url{http://nvdla.org/primer.html}.
\bibitem[NVIDIA(2023)]{nvidia_h100_tensor_core_gpu}
NVIDIA.
\newblock Nvidia h100 tensor core gpu.
\newblock \url{https://www.nvidia.com/en-sg/data-center/h100/}, 2023.
\newblock Accessed: 2023-12-12.
\bibitem[Paperno et~al.(2016)Paperno, Kruszewski, Lazaridou, Pham, Bernardi, Pezzelle, Baroni, Boleda, and Fern{\'a}ndez]{paperno2016lambada}
Denis Paperno, Germ{\'a}n Kruszewski, Angeliki Lazaridou, Quan~Ngoc Pham, Raffaella Bernardi, Sandro Pezzelle, Marco Baroni, Gemma Boleda, and Raquel Fern{\'a}ndez.
\newblock The lambada dataset: Word prediction requiring a broad discourse context.
\newblock \emph{arXiv preprint arXiv:1606.06031}, 2016.
\bibitem[Paszke(2019)]{paszke2019pytorch}
A~Paszke.
\newblock Pytorch: An imperative style, high-performance deep learning library.
\newblock \emph{arXiv preprint arXiv:1912.01703}, 2019.
\bibitem[Sakaguchi et~al.(2021)Sakaguchi, Bras, Bhagavatula, and Choi]{sakaguchi2021winogrande}
Keisuke Sakaguchi, Ronan~Le Bras, Chandra Bhagavatula, and Yejin Choi.
\newblock Winogrande: An adversarial winograd schema challenge at scale.
\newblock \emph{Communications of the ACM}, 64\penalty0 (9):\penalty0 99--106, 2021.
\bibitem[Sap et~al.(2019)Sap, Rashkin, Chen, LeBras, and Choi]{sap2019siqa}
Maarten Sap, Hannah Rashkin, Derek Chen, Ronan LeBras, and Yejin Choi.
\newblock Socialiqa: Commonsense reasoning about social interactions.
\newblock \emph{arXiv preprint arXiv:1904.09728}, 2019.
\bibitem[Stevens et~al.(2021)Stevens, Venkatesan, Dai, Khailany, and Raghunathan]{stevens2021softermax}
Jacob~R Stevens, Rangharajan Venkatesan, Steve Dai, Brucek Khailany, and Anand Raghunathan.
\newblock Softermax: Hardware/software co-design of an efficient softmax for transformers.
\newblock In \emph{2021 58th ACM/IEEE Design Automation Conference (DAC)}, pp.\ 469--474. IEEE, 2021.
\bibitem[Tekgurler(2025)]{translation}
Merve Tekgurler.
\newblock Llms for translation: Historical, low-resourced languages and contemporary ai models.
\newblock \emph{arXiv preprint arXiv:2503.11898}, 2025.
\bibitem[Touvron et~al.(2023)Touvron, Lavril, Izacard, Martinet, Lachaux, Lacroix, Rozi{\`e}re, Goyal, Hambro, Azhar, et~al.]{touvron2023llama}
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timoth{\'e}e Lacroix, Baptiste Rozi{\`e}re, Naman Goyal, Eric Hambro, Faisal Azhar, et~al.
\newblock Llama: Open and efficient foundation language models.
\newblock \emph{arXiv preprint arXiv:2302.13971}, 2023.
\bibitem[Varghese \& Sambath(2024)Varghese and Sambath]{varghese2024yolov8}
Rejin Varghese and M~Sambath.
\newblock Yolov8: A novel object detection algorithm with enhanced performance and robustness.
\newblock In \emph{2024 International Conference on Advances in Data Engineering and Intelligent Computing Systems (ADICS)}, pp.\ 1--6. IEEE, 2024.
\bibitem[Workshop et~al.(2022)Workshop, Scao, Fan, Akiki, Pavlick, Ili{\'c}, Hesslow, Castagn{\'e}, Luccioni, Yvon, et~al.]{workshop2022bloom}
BigScience Workshop, Teven~Le Scao, Angela Fan, Christopher Akiki, Ellie Pavlick, Suzana Ili{\'c}, Daniel Hesslow, Roman Castagn{\'e}, Alexandra~Sasha Luccioni, Fran{\c{c}}ois Yvon, et~al.
\newblock Bloom: A 176b-parameter open-access multilingual language model.
\newblock \emph{arXiv preprint arXiv:2211.05100}, 2022.
\bibitem[Xiao et~al.(2023)Xiao, Lin, Seznec, Wu, Demouth, and Han]{xiao2023smoothquant}
Guangxuan Xiao, Ji~Lin, Mickael Seznec, Hao Wu, Julien Demouth, and Song Han.
\newblock Smoothquant: Accurate and efficient post-training quantization for large language models.
\newblock In \emph{International Conference on Machine Learning}, pp.\ 38087--38099. PMLR, 2023.
\bibitem[Yu et~al.(2022)Yu, Park, Park, Kim, Lee, Lee, and Choi]{yu2022nn}
Joonsang Yu, Junki Park, Seongmin Park, Minsoo Kim, Sihwa Lee, Dong~Hyun Lee, and Jungwook Choi.
\newblock Nn-lut: Neural approximation of non-linear operations for efficient transformer inference.
\newblock In \emph{Proceedings of the 59th ACM/IEEE Design Automation Conference}, pp.\ 577--582, 2022.
\bibitem[Zellers et~al.(2019)Zellers, Holtzman, Bisk, Farhadi, and Choi]{zellers2019hellaswag}
Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi, and Yejin Choi.
\newblock Hellaswag: Can a machine really finish your sentence?
\newblock \emph{arXiv preprint arXiv:1905.07830}, 2019.
\bibitem[Zhang et~al.(2022)Zhang, Roller, Goyal, Artetxe, Chen, Chen, Dewan, Diab, Li, Lin, et~al.]{zhang2022opt}
Susan Zhang, Stephen Roller, Naman Goyal, Mikel Artetxe, Moya Chen, Shuohui Chen, Christopher Dewan, Mona Diab, Xian Li, Xi~Victoria Lin, et~al.
\newblock Opt: Open pre-trained transformer language models.
\newblock \emph{arXiv preprint arXiv:2205.01068}, 2022.
\bibitem[Zhao et~al.(2024)Zhao, Lv, Xu, Wei, Wang, Dang, Liu, and Chen]{zhao2024detrs}
Yian Zhao, Wenyu Lv, Shangliang Xu, Jinman Wei, Guanzhong Wang, Qingqing Dang, Yi~Liu, and Jie Chen.
\newblock Detrs beat yolos on real-time object detection.
\newblock In \emph{Proceedings of the IEEE/CVF conference on computer vision and pattern recognition}, pp.\ 16965--16974, 2024.
\bibitem[Zong \& Krishnamachari(2022)Zong and Krishnamachari]{gpt3}
Mingyu Zong and Bhaskar Krishnamachari.
\newblock A survey on gpt-3.
\newblock \emph{arXiv preprint arXiv:2212.00857}, 2022.
\end{thebibliography}