Generalized end-product feedback circuit senses high dimensional environmental fluctuations

Extracted source text · arXiv 2211.12614

\documentclass[%
 reprint,
%superscriptaddress,
%groupedaddress,
%unsortedaddress,
%runinaddress,
%frontmatterverbose, 
%preprint,
%preprintnumbers,
%linenumbers,
nofootinbib,
%nobibnotes,
%bibnotes,
 amsmath,amssymb,
 pre,
%pra,
%prb,
%rmp,
%prstab,
%prstper,
%floatfix,
]{revtex4-2}
%\usepackage[utf8]{inputenc}
\usepackage{graphicx}
\usepackage{float}

\begin{document}
\title{Generalized end-product feedback circuit senses high dimensional environmental fluctuations}

\author{Fang Yu}
\author{Mikhail Tikhonov}
\date{May 2022}


\begin{abstract}
   Understanding computational capabilities of simple biological circuits, such as the regulatory circuits of single-cell organisms, remains an active area of research. Recent theoretical work has shown that a simple regulatory architecture based on end-product inhibition can exhibit predictive behavior by learning fluctuation statistics of one or two environmental parameters. Here we extend this analysis to higher dimensions. We show that as the number of inputs increases, a generalized version of the circuit can learn not only the dominant direction of fluctuations, as shown previously, but also the subdominant fluctuation modes.
\end{abstract}

\maketitle

\section{Introduction}
As organisms evolve to better survive in changing environments, they develop adaptations that allow them to respond to change, but also to predict change. Characterizing such predictive (anticipatory \cite{Tagkopoulos, Savageau}) behavior in microorganisms, whose regulatory architectures are far less complex than can be achieved by a neuron-based brain, revealed many examples of evolutionary ingenuity attaining complex objectives with minimal ingredients (e.g., robust circadian clocks in photosynthetic algae, which allow them to reorganize their metabolism in preparation for sunrise~\cite{Pedersen}). 

Theoretical computer science has long established that even the simplest building blocks, if used in sufficient numbers, can support complex computations: very simple instruction sets can already be Turing-complete~\cite{Turing, Brainerd, Perez}. The biologically relevant sister question---how simple of a \emph{circuit} can perform how complex a task?---is understood less well. Some well-studied examples include the chemotaxis circuit achieving perfect adaptation \cite{Yi, Chang}, mechanisms of temperature compensation in circadian clocks \cite{Pedersen, Husain}, or the bistable genetic regulatory network storing and retrieving associative memories \cite{Sorek}. Still, understanding the computational capabilities of biological circuits remains an active area of research. 

Recent theoretical work explored the ability of a simple circuit to learn statistical features of fluctuating environmental parameters \cite{Stefan}. It was shown that this task can be solved (in fact, near-optimally) by a simple implementation based on the end-product inhibition motif. That work considered the proof-of-principle cases of one or two environmental parameters, sufficient to demonstrate how the circuit can learn, respectively, the variances and correlations of its inputs and to discuss implementations with biologically plausible ingredients. However, optimization tasks solved by cells (e.g.\ resource allocation) are often high-dimensional. It is natural to ask whether the same architecture would retain its performance when the number of fluctuating factors increases. 

Here, we explore a high-dimensional generalization of the circuit proposed in Ref.~\cite{Stefan}. We ask whether the three ingredients identified in that work---nonlinearity, an excess of regulators, and cross-talk between them---are sufficient to learn the fluctuation structure of high-dimensional environments. 

We find that, as in the low-dimensional case, the circuit can upregulate its reactivity to respond faster in epochs when environment fluctuations are larger. In principle, this allows it to outperform simple end-product inhibition at the tracking task defined in \cite{Stefan}, but we argue that in the high-dimensional case this small performance gain is unlikely to justify the significant complexity cost. However, we further show that even a small excess of regulators already makes the circuit responsive to changes in fluctuation structure, and that the state adopted by the circuit encodes both the dominant and subdominant fluctuation modes of environmental parameters. We conclude that the generalized end-product feedback circuit could serve as a sensor of subtle environmental change.

%Regulatory process, in which regulators detect the dynamically changing environment and response to it, is ubiquitous in biological systems across different scales. For example, enzymes detect physio-chemical changes in their metabolic environment while their expression promotes or inhibits the production or consumption of metabolites, which in turn affects a number of physio-chemical properties. 

%One way to adapt to changing environment is through predictive adaptation. A predictive adaptation architecture allows regulators to combine their observations from past environmental fluctuations to make predictions, and adjust their expression level accordingly. For example, circadian clocks allow photosynthetic algae to reorganize their metabolism in preparation for the rising sun \cite{Pedersen, Husain}. Another example is the anticipatory behavior in E. coli, which allows it to prepare for the next environment under its normal cycling through the mammalian digestive tract \cite{Savageau}.

% It was recently proposed that a simple architecture can exhibit predictive adaptation \cite{Stefan}. It includes extra regulators, nonlinearity in the regulator activity, and cross talks among regulatory pathways. With these key ingredients, this generalized end-product feedback architecture 'learns' correlations of environmental fluctuations near-optimally in 2-dimensional case.

% A well-known example regulatory motif of predictive adaptation is simple end-product inhibition. In a simple end-product inhibition, a regulator is making prediction to track some external environmental factor. A feedback inhibition improves its prediction by inhibiting its expression if it over-predicts, or promotes its expression if it under-predicts.

% However, simple end-product inhibition can only track one single resource per regulator and is not reactive to correlated environmental fluctuations. This is because no matter how many regulators there are, the expression of them would always depend on the same number of regulators one-one correspondingly, and are therefore always independent. On the other hand, environmental fluctuations in the real world can be highly correlated. In these environments, SEPI would fail to predict their fluctuation structure, which calls for the need of a better suitable regulatory system. How can a regulatory system track not only each individual fluctuating component but also the correlation structure among them to make better predictions?

% Prior opinions have pointed out the probability that the evolution of ecological organisation produces intelligent design-like feature of learning via nonlinear correlation\cite{Watson}. For example, genetic regulatory network learn subtle binary features, such as a (transient) presence or absence of a correlation between two signals\cite{Sorek}. Stefan Landmann et. al's work\cite{Stefan} considered nonlinear correlation in the design of regulatory network model and proposed a generalization of end-product inhibition architecture, and shown that it can 'learn' correlations of environmental fluctuations near-optimally in 2-dimensional case.

% However, vital difference lies between two dimensional and higher dimensional cases: in two dimensional case, there is only one pair of correlation. Therefore, learning the fluctuation structure simply means learning that pairwise correlation; But in higher dimensional case, much more subtle and complex correlation structures are to be learned. These key differences naturally give rise to the questions: Can this generalized end-product feedback architecture learn the fluctuation structure in higher dimensions? If so, how many regulators and how much cost is needed? We are addressing these questions in this paper.

% To address these questions, we extended the generalized end-product feedback architecture to high dimensions. We looked at its sensitivity to environmental change and alignment to environmental fluctuation structures. We show that this architecture can learn high dimensional fluctuation structures. Moreover, our architecture serves as a sensor of the subtle statistics feature, such as the sub-leading correlations among environmental fluctuations.

\section{The model}
Our approach builds directly on that of Landmann et al.~\cite{Stefan}, but this section provides enough details to be self-contained.

Specific adaptation problems faced by real organisms are highly diverse. Following Ref.~\cite{Stefan}, here we distil the general problem of physiological learning to a minimal model. Specifically, we consider a scenario where a set of internal quantities $\vec{P} = (P_1,\dots,P_N)$ (which a cell can regulate) must track a set of fluctuating external factors, $\vec{D} = (D_1,\dots,D_N)$. For the sake of concreteness, we will think of this problem in metabolic terms, with $P_i$ representing the rates of production of metabolites $x_i$. In our model, the cell seeks to match these production rates $\vec P$ to the (time-dependent) demands $\vec{D}(t)$ imposed by the external conditions. As an example, environmental conditions that trigger biofilm formation in bacteria require a different stoichiometry of synthesis than the condition of fast planktonic growth. 

If the fluctuations of demands are slow, the organism could sense them and directly match $\vec P$ to $\vec D$ at all times. But if fluctuations are too fast to be followed precisely, the organism must instead rely on the ``statistical structure'' of $\vec D(t)$, such as the mean value or correlations between its components $D_i(t)$~\cite{Stefan}. If this statistical structure remains constant over a very long timescale, the optimal behavior (given this structure) could be hardwired into the circuit by evolution. But if the structure itself occasionally changes, the organism would need to learn it from recent observations via physiological mechanisms. This is the regime where this problem can serve as a minimal model for the task of physiological learning.

% [Now several subsections building up the setup by considering different forms of statistical structure]

There are different levels of statistical structure to be learned. Under our tracking problem, the simplest form of learning would be to set the production rates $P_i$ to match the average demand in the recent past. Beyond that, the subtler statistics include the variances and correlations among fluctuations. To model $D(t)$ in a way where both means and correlations can be tuned, we consider a multi-dimensional random walk in a quadratic potential~\cite{Stefan}.
% The simplest level is the signal mean: a single value for each component of $\vec D$. We expect an organism capable for learning to first see through the rapid fluctuations and store the relatively stable, slowly-changed average of them into its memory

\begin{equation}
    \vec{D}(t+\Delta t) = \vec{D}(t) - M\Delta t\cdot(\vec{D}(t) - \vec{\langle D\rangle}) + \sqrt{2\Gamma\Delta t}\,\vec{\eta}\label{eq:1}
\end{equation}
Here $\vec{\langle D\rangle}$ denotes the average demand, $\Gamma$ denotes the fluctuation strength, and $\vec{\eta}$ is a series of independent Gaussian random variables with zero mean and unit variance. The matrix $M$ determines the correlation among fluctuations of different components of $D$. If $M$ is isotropic, the fluctuations of individual components of $D$ will be decoupled.

Our approach will be as follows. To probe whether a given regulatory architecture successfully learns the statistical structure of the fluctuating environment, we expose it in simulations to several environmental epochs that differ by statistical structure. To say that the system successfully ``learns'' its environment, we require two criteria. First, the system should be sensitive to the change of statistics, i.e.\ we expect the regulator activity to be reorganized between epochs. Second, we should be able to exhibit the ``rule'' by which the statistical feature of interest is encoded in regulator activity. 

%Simplest case: mean, can already be achieved with SEPI
The simplest form of statistical structure is the average demand $\vec{\langle D\rangle}$. This average demand can be learned already by the simple end-product inhibition (SEPI) circuit, where the production $P_i$ of each metabolite $x_i$ is placed under control of a single dedicated regulator $a_i$ inhibited by $x_i$ itself (Fig.~\ref{fig:1}A). In epochs of low demand, the unused $x_i$ accumulates and decreases production until it balances the demand. The average demand over a recent past is stored in the activity of the regulator $a_i$ (Fig.~\ref{fig:1}B, C).
% [Fig 1 discussed here]. [Refer to SI as showing that a is indeed the average over recent past; this is also where all the equations go.]

\begin{figure}[h!]
\centering
  \includegraphics[width=0.4\textwidth]{f0.png}
  \caption{The SEPI architecture can be seen as learning signal mean. (A) Simple end-product inhibition (SEPI). (B) We expose the SEPI architecture to 3 environmental epochs (solid line) that differ by the average demand $\vec{\langle D\rangle}$ (dashed line). (C) The expression level of the regulator $a$ encodes the average demand. }
  \label{fig:1}
\end{figure}


The regulatory motif of end-product inhibition is not only simple, but has been shown to be remarkably effective. For example, under certain assumptions, this motif alone can not only ``solve'' the problem of proteome reallocation after a change of environmental conditions, but do so in an optimal time \cite{Pavlov}. However, the effectiveness of SEPI necessarily applies only when dealing with states that, in the language of our model, differ by the signal mean. The internal degrees of freedom (the regulators $a$, serving as memory) can store only one value per metabolite $x_i$. To be sensitive to additional statistics, additional degrees of freedom would necessarily be required. Thus, from now on, we will allow the number of regulators $N_a$ to exceed the number of metabolites $N_x$, and label regulators using Greek indices $\mu$, running from 1 to $N_a$ (while Roman indices $i$, labeling metabolites, run from 1 to $N_x$).

Landmann et al.\ showed that a generalized end-product feedback architecture can learn the variances and correlations of $D_i$~\cite{Stefan}. Their architecture takes three ingredients: an excess of regulators ($N_a > N_x$), nonlinear activation/repression of the regulators $a_\mu$ by the metabolite concentrations $x_i$, and cross-talk among different regulatory pathways. Specifically, they considered the following dynamics:

\begin{equation}
    \begin{aligned}
    x_i &= \frac{P_i}{D_i} \\
    P_i &= \sum_\mu\sigma_{\mu i}a_\mu \\
    \tau_a \dot{a}_\mu &= a_\mu \max\Big(d,\sum_i\sigma_{\mu i}(1 - x_i)\Big) - \kappa a_\mu \\
    \end{aligned}
    \label{eq:2}
\end{equation}
Here $\sigma_{\mu i}$ describes how the activities of regulators $a_\mu$ control the synthesis of metabolites $x_i$; $d$ parameterizes nonlinearity; and $\kappa$ is the degradation rate.

\begin{figure*}[t!]
  \includegraphics[width=0.95\textwidth]{F1_2.png}
  \caption{The generalized architecture is built to learn the variances and correlations of demand $\vec{D}$. (A) An example of generalized end-product feedback architecture in the case where $N_x=3$ and $N_a=4$. (B) 4 regulatory pathways on 3 resources are shown as 4 vectors in 3d space. Regulation vectors are simulated as charges repelling each other to distribute evenly. (C) To test the ability of the system to learn, we expose it to 3 environmental epochs overtime differed in the dominant eigenvector of $M$. shown here ellipsoids representing the orientation of the eigenbasis of $M$. (D) In response to these changes, the system dynamically adjusts the expression level of regulators. }
  \label{fig:2}
\end{figure*}

In two dimensions ($N_x=2$), this circuit can sense, store and usefully ``recall'' the information on second-order input  statistics, such as variances and correlations, and do so near-optimally~\cite{Stefan}. Here, we extend this architecture to higher dimensions (Fig.~\ref{fig:2}A). We choose regulators $\sigma$ to be minimally redundant (see SI section B). Briefly, the elements of $\sigma$, normalized as $|\sigma|=1$, can be seen as points on a sphere, and we pick them to be spread out as far away from each other as possible by treating them as repelling charges on a sphere's surface (Fig.~\ref{fig:2}B). We expose the generalized end-product feedback architecture to environmental epochs that differ in fluctuation structure $M$ only. To guarantee that any restructuring of regulator activity between exposure epochs is due to the changes of $M$, we keep the mean demand $\bar D$ the same in all epochs. For concreteness, we pick $M$ to be a random rotation of $\left(\begin{smallmatrix}
  1 & 0 & \cdots & 0\\
  0 & 100 & \cdots & 0\\
  \vdots & \vdots & \ddots & \vdots\\ 
  0 & 0 & \cdots & 100
\end{smallmatrix}\right)$, so that the environmental fluctuations have one preferred direction. When shifting from one epoch to another, we reorient $M$ by applying a random rotation (Fig.~\ref{fig:2}C), and observe how the regulatory architecture reorganizes the regulator expression levels in response (Fig. \ref{fig:2}D).

Note that changing the direction of the dominant eigenvector is only one way to change the environment. For example, Ref.~\cite{Stefan} also considered environments with different extent of correlation among fluctuations of $D_i$'s, which the circuit was also able to learn. Here, we will use the former approach, because the existence of a preferred direction of fluctuations allows for more visual metrics to quantify how well the circuit learned, as we describe below. We will show that the architecture of Fig.~\ref{fig:2}A is indeed responsive to higher-dimensional rotations of $M$, and will quantify this sensitivity. 
% For example, the angle between the true dominant direction and the circuit-encoded estimate, namely the dominant eigenvector of the responsiveness matrix $R_{ij}=\frac{d\dot{P_i}}{dD_j}$.
 
\section{Results}
\subsection{The generalized architecture\\ can outperform SEPI, but is costly}

When Landmann et al considered the architecture (Eq~\ref{eq:2}) in the two-dimensional case, their focus was not just learning, but also the benefit of learning. Specifically, their primary readout was the tracking performance defined by $-\sqrt{\sum_i(P_i-D_i)^2}$, a proxy for organism fitness, and they showed that the learning-capable circuit can enhance tracking performance over SEPI. In this section, we demonstrate that this observation continues to hold in higher dimensions: namely, the circuit of Fig.~\ref{fig:2}A can achieve better tracking performance than the SEPI architecture. However, we will also show that this performance increase is very costly.

% Describe Fig.~3 here. Be concise: what's on the axes, and the two measures of cost.

Here we use two metrics of cost: Control input power (CIP), defined by $\int{||\dot{P}||^2dt}$, is a concept borrowed from control theory. Measuring cost in this way has the advantage that the family of optimal strategies on the performance-CIP plane can be derived analytically; but CIP is not easy to interpret in biological terms. A more biologically relevant measure of cost is the total expression of all regulators combined: $\sum_\mu a_\mu$. 
% For example, in metabolic context, expression of extra metabolites cost energy and the most economic way is to maintain regulatory pathways via expressing as few regulators as possible.

% Comparing the performance of our architecture to the best strategy curve, we show that our architecture performs near-optimally when exposing to correlated high-dimensional fluctuating environment. However, comparing the performance and cost of our architecture with those of SEPI, We find that improving performance beyond SEPI is very costly. SEPI is already good, achieving a near-optimal overall performance. Below the SEPI performance, one can improve the performance of our architecture by adding more regulators, but above that value, adding more regulators does not affect much, on the other hand, much more cost is required (Fig.~\ref{fig:3}). This can be achieved only by changing the parameters in regulatory pathways. For example, by reducing the leaking rate $\kappa$.




\begin{figure}[t!]
  \includegraphics[width=\linewidth]{F2.png}
  \caption{Improving performance beyond SEPI is very costly. (A) Tracking performance of different architectures when $N_x=6$, shown against Control Input Power (CIP), which is a measure of cost (see text). Red dots show simulation results of the optimal strategy, and the red line is the smoothed curve. The blue dot indicates SEPI performance, and the grey line shows the performance of our architecture, with dots changing from blue to red indicating an increasing number of regulators. (B) Same as A, replotted using a more biologically relevant measure of cost (the total expression of all regulators $\sum_\mu a_\mu$). }
  \label{fig:3}
\end{figure}

Fig.~\ref{fig:3} confirms that both cost metrics yield similar results. As expected, increasing the number of regulators increases performance. Performance can also be improved by reducing the degradation rate $k$, since a higher expression of regulators (with activators and repressors active simultaneously, known as paradoxical regulation~\cite{Hart}) allows $P_i$ to change faster~\cite{Stefan}. As a result, the performance of the generalized architecture can exceed that of SEPI. However, Fig.~\ref{fig:3} also shows the remarkable effectiveness of SEPI, which lies closest to the optimal curve at minimal circuit complexity. As described in Ref.~\cite{Stefan}, the advantage of the architecture (Eq.~\ref{eq:2}) is not to outperform SEPI in the absolute sense of the Pareto front of performance vs.\ cost. Rather, it provides a way of investing extra resources into improving performance at some important task. Such mechanisms are known to be employed by cells in other contexts, e.g. investing energy to improve the accuracy of sensing~\cite{Laughlin,Lan} or copying its DNA~\cite{Olson}. However, in our context it seems implausible that this marginal performance increase alone would be sufficient to offset the cost of a significant increase in protein expression levels and circuit complexity. Thus, from here on, we will no longer consider tracking performance as our readout. Instead, we will assume that an ability to sense subtle changes in environmental statistics may itself be of value to the organism (e.g.\ as an early cue indicative of some other upcoming change), and investigate the ability of this circuit to learn the environmental state and react to its changes.

\subsection{The architecture is sensitive to and tracks the dominant eigenvector of M}

\begin{figure}[b!]
  \includegraphics[width=\linewidth]{F3-3.png}
  \caption{A modest excess of regulators allows the generalized architecture to learn. (A) Heatmap of sensitivity $\chi$ shows that the generalized architecture responds to environmental change with an excess of regulators. (B) Heatmap of the alignment $\gamma$ between responsiveness $R$ and the dominant eigenvector of $M$ indicates that with a modest excess of regulators, the generalized architecture tracks the dominant eigenvector of $M$ by aligning its responsiveness towards it. }
  \label{fig:4}
\end{figure}

% Explanation of what 'learning fluctuation structures' means.
% Learning fluctuation structures means aligning the direction of responsiveness of the system with the statistical correlations of its external environmental factors. Responsiveness is defined as $R_{ij} = \frac{d\dot{P}_i}{dD_j}$. One can imagine that in a good case of learning, the regulatory network intelligently assigns its steering rate of predictions on different environmental components according to change of each external component so that they track the varying rate of external factors correlated to change of per factor, which is on average exactly the recovery potential matrix $M$ in the fluctuation formula. In such case $R$ would track $M$. The better aligned the responsiveness is to M, the better learning a regulatory network achieves.

To `learn' fluctuation structure, our architecture must first be sensitive to it. This means that when environmental statistics change, the expression level of regulators should change. To quantify this, we compare the expression levels before and after each environmental change, and define the sensitivity $\chi$  as the average angle between them  (interpreted as vectors in the $N_a$-dimensional expression space): 
\[
\chi=\frac 1{K-1}\sum_{k=1}^{K-1}\arccos \left[\frac{\vec{a}^{\,(k)}\cdot\vec{a}^{\,(k+1)}}{\|\vec{a}^{\,(k)}\|\,\|\vec{a}^{\,(k+1)}\|}\right].
\] Here $K=10$ is the number of environment epochs, and $\vec a^{\,(k)}$ denotes the average expression of all regulators in epoch $k$ (estimated by averaging the fluctuating expression over the second half of the epoch, when the expression reaches steady state; compare to Fig. 2D). Fig.~\ref{fig:4}A confirms that the architecture (Eq.~\ref{eq:2}) responds to the change in environmental statistics (Eq.~\ref{eq:1}), and is increasingly sensitive to it as the number of regulators increases.

We will now show that the regulation state adopted by the system is not random or idiosyncratic, but encodes information about $M$ in a simple way. Namely, we will show that the system preferentially aligns its responsiveness, $R_{ij} = \frac{d\dot{P}_i}{dD_j}$, to the dominant eigenvector of $M$. This behavior is, in fact, the ``smart'' thing to do: it can be shown that the optimal strategy (in the sense of control theory, with CIP as cost metric) would be similarly anisotropic, with the dominant direction of fluctuations eliciting the strongest response~\cite{Stefan, Liberzon}. To quantify this degree of alignment, we define \[
\gamma=\mathrm{Prob}_{\|u\|=1}\Big(\|R\cdot \vec{u}\|>\|R\cdot \vec{v}_M\|\Big)
\]
where $v_M$ denotes the dominant eigenvector of $M$. This metric quantifies the probability that a randomly drawn unitary vector $\vec{u}$ is aligned as good or better than $v_M$ to the responsiveness $R$. Unlike sensitivity $\chi$'s which are angles measured in increasingly high-dimensional spaces and are therefore affected by dimensional effects, $\gamma$ is defined to be comparable across dimensions. The smaller the value of $\gamma$, the stronger the evidence that our generalized end-product feedback architecture adopted a state with a responsiveness matrix preferentially aligned to $v_M$.
% When $Na = Nx + 2$, the responsiveness $R$ of our architecture is better aligned to the dominant eigenvector of $M$ than 80\% of the random samples; when $Na > Nx + 2$, this fraction approaches or exceeds 90\%. Therefore, the regulatory state adopted by the generalized end-product feedback architecture successfully tracks the dominant eigenvector of M.
Heatmap of $\gamma$ (Fig.~\ref{fig:4}) shows that even just two excess regulators exhibit clear signature of the architecture aligning better than random.

% Learning the fluctuation structure may also require some cost. To measure this, we show the normalized alignment angle $\theta$ vs control input power and vs the total expression of regulators. 
% Another intuitive metric of the architecture's alignment ability is the angle $\theta$ between the dominant eigenvectors of responsiveness $R$ and fluctuation statistics $M$. To make the values comparable, we measure how rare $\theta$ is by quantifying the probability that it is smaller than an angle between the dominant eigenvector of $M$ and a random vector in the same dimension. (The same method as we did to $\gamma$). For simplicity, here we use it only in showing the cost required for the architecture's learning the fluctuation structure (Fig.~\ref{fig:5}), while more plots can be found in SI section C.
Learning the fluctuation structure may also require some cost. To measure this, we show $\gamma$ vs CIP and the total expression level of regulators ($\sum a_\mu$) in Fig.~\ref{fig:5}.
% Definition of $\theta$ and how we normalize it.
% learning of the fluctuation structure can be significantly improved by having more regulators without extra cost
In contrast to Fig.~\ref{fig:3}, Fig.~\ref{fig:5} shows that the learning ability of the circuit is controlled primarily by the number of regulators instead of degradation rate, and can be modulated without incurring an expression or CIP cost.
% The results show that the normalized alignment angle $\theta$ can be significantly reduced by having more regulators without extra cost (Fig.~\ref{fig:4}). Therefore, increasing the number of regulators is a non-costly way to improve the architecture's learning of fluctuation structures.

\begin{figure}[t!]
  \includegraphics[width=0.45\textwidth]{F4_gamma.png}
  \caption{More regulators enhance learning of fluctuation structure without extra cost. (A) $\gamma$ (alignment between responsiveness $R$ and the dominant eigenvector of $M$) vs CIP in the case of $N_x = 6$ for different $N_a$ and $\kappa$. (B) $\gamma$ vs CIP in the same condition. The color of dots changing from blue to red indicates an increasing number of regulators.}
  \label{fig:5}
\end{figure}
% Normalized alignment angle $\theta$ vs CIP when Nx = 6 for different Na and $\kappa$.  (B) Normalized $\theta$ vs sum of $a_\mu$'s in the same condition. The color of dots changing from blue to red indicates an increasing number of regulators.

\subsection{The architecture also tracks non-dominant statistics}

\begin{figure*}[t!]
  \includegraphics[width=\linewidth]{F5_2.png}
  \caption{Our architecture learns to track the changing sub-dominant eigenvector of the input fluctuation. (A) We rotate $M$ with its dominant eigenvector fixed. (B) Heatmap of the rotation angle of vector $\vec{a} = \{a_1,\dots, a_\mu,\dots, a_{Na}\}$ indicates that our architecture is sensitive to changing subdominant eigenvector of $M$.  (C) Heatmap of the alignment $\gamma$ between responsiveness $R$ and the dominant eigenvector of $M$. (D) Heatmap of the alignment $\digamma$ between non-dominant eigenvectors of $R$ and $M$.}
  \label{fig:6}
\end{figure*}

Till now we have been considering the case where $M$ has only one dominant direction, but what happens when $M$ is more complex? To test this, we consider $M$ with a sub-dominant eigenvector---specifically, a randomly rotated version of $M_0=\left(\begin{smallmatrix}
  1 & 0 & 0 & \cdots & 0\\
  0 & 2 & 0 & \cdots & 0\\
  0 & 0 & 100 & \cdots & 0\\
  \vdots & \vdots & \vdots & \ddots & \vdots\\ 
  0 & 0 & 0 & \cdots & 100
\end{smallmatrix}\right)$---and restrict environment changes to those that change only the sub-dominant direction, keeping the dominant eigenvector fixed. For example, in a three-dimensional case, one can intuitively think of this as rotating an anisotropic ellipsoid around its dominant axis (Fig.~\ref{fig:6}A). Fig.~\ref{fig:6}B confirms that for any $N_x>2$, the system indeed alters its regulatory state in response to such environmental changes. The $N_x=2$ column serves as a control: in two dimensions, the only rotation that preserves the dominant eigenvector is the identity matrix. Thus, the environmental statistics do not change between epochs, and any observed differences in ``regulatory state'' are due to the noise in our estimation of average expression from finite simulation data (averaging the fluctuating expression over a finite simulation time). We observe that as long as $N_a\geq N_x+2$, the changes of regulatory state exceed the noise floor, confirming that the system is sensitive not only to the dominant direction of fluctuations (cf. Fig~\ref{fig:4}A), but to the direction of the second eigenvector as well. 

How well does the system learn? Plotting gamma indicates that the alignment to the dominant eigenvector (Fig.~\ref{fig:6}C) remains significant (better than random), but becomes worse than we observed in Fig.~\ref{fig:4}B. This is, of course, expected: the structure of fluctuations no longer reduces to a single dominant direction, so when assessing the alignment between the responsiveness $R$ and $M$, looking at only the dominant eigenvector of $M$ is insufficient. To also take non-dominant eigenvectors of $M$ into consideration, we defined the alignment $\phi$ between the whole matrices $R$ and $M$ as \[\phi_{[M,R]} = \frac{\|[M,R]\|}{\|\{M,R\}\|}\]

% And the cross-dimensional comparable metric of the alignment between non-dominany eigenvector of responsiveness $R$ and $M$ as \[\digamma=Prob_{R'}(\phi_{[M,R']}<\phi_{[M,R]})\]

Where $\|\dots\|$ denotes the Frobenius norm of a matrix. If $M$ and $R$ are diagonal in the same basis, they commute and $\phi$ is zero. Thus, a non-zero $\phi$ can be seen as a measure of misalignment between the eigenbasis of $M$ and $R$.

Similar to the trick we did to $\gamma$, to enable meaningful comparisons across dimensions, instead of focusing on the raw value of $\phi$, we compute the probability \[\varphi=\mathrm{Prob}_{R'}(\phi_{[M,R']}<\phi_{[M,R]}).\]

Here $R'$ is a random rotation to $R$ with its dominant eigenvector fixed. If $\varphi$ is close to zero, it means that [...].

% Therefore, [...] We take the alignment between the whole matrices $R$ and $M$ into consideration.

% Explain FY metric.
% This alignment metric is defined by what fraction of alignment between $M$ and a random dominant-eigenvector-fixed rotation to $R$ has higher $\sqrt{\frac{\sum_{\mu i}[M,R]^2}{\sum_{\mu i}\{M,R\}^2}}$ than that between $R$ and $M$. Intuitively, if responsiveness $R$ aligns perfectly to $M$. $M$ and $R$ should commute with each other. Therefore, we take the norm of the commutator $||[M,R]||$ and normalize it by the norm of the anti-commutator $||\{M,R\}||$ so that smaller $\frac{||[M,R]||}{||\{M,R\}||}$ indicates better alignment in most of the parameter range. By ranking it among that of $M$ and a random dominant-eigenvector-fixed rotation to $R$, we show how rare the alignment between responsiveness $R$ of our architecture and $M$ is.

Fig.~\ref{fig:6}D shows that particularly in dimensions 3 and 4, the generalized architecture (Eq.~\ref{eq:2}) is not only responsive to changes in subdominant direction of fluctuations, but succeeds at realigning its responsiveness matrix accordingly.
% But even when there are multiple environmental components, the signal of our architecture learning the non-dominant statistics is still obvious. This provide solid evidence that our architecture 'intelligently' adjusts its expression level of regulators to learn not only the dominant correlation statistics of the external fluctuations, but also those more subtle, less dominant ones.

\section{Discussion}

% This illustrates that distinction between "reactive" and "predictive" systems is a blurry one. The end-product inhibition is a purely reactive circuit that upregulates production in response to an increased demand. But if the demand (the "signal") takes the shape of fast fluctuations over a slowly changing mean, the current estimate of this mean is the best predictor of the future value of the signal, and so the circuit can be seen as making a simplest form of ``prediction''. The internal degree of freedom (the activity level of the regulator a) serves as a memory, updated as new evidence arrives.

The regulatory circuit we considered in this work generalizes simple end-product inhibition by including three additional ingredients: nonlinearity, an excess of regulators, and cross-talk between them. Previous work has shown that these ingredients can endow the circuit with an ability to learn time-dependent fluctuation statistics of its inputs through a form of associative learning, at least in the low-dimensional scenarios (with one or two inputs)~\cite{Stefan}. Here, we generalized this circuit to the higher-dimensional case and presented two results. Just like in lower dimensions, this architecture can show an improved performance at the task of tracking environmental fluctuations; but we have argued that this small performance gain is unlikely to offset the significant complexity cost in practice. However, if sensing changes in environmental statistics is of value to the organism, then this architecture is quite interesting as it offers a sensitivity to subtle changes, sensing not only the dominant direction of fluctuations, but also the subdominant fluctuation modes. 

How relevant is this high-dimensional case for real cells? It is easy to imagine that a specific pair of resources might be correlated at some point of an organism's lifecycle but not at another; thus, it is clear that dynamically learn and unlearn a \emph{single} correlation could be useful. By comparison, the ``universal learning'' capacity described here---an ability to pick out any preferred direction of correlations in a high-dimensional space---seems rather more abstract. However, this could be the relevant regime for the regulatory architecture of a cell as a whole, seen as a high-dimensional learning circuit. Some intriguing recent ideas propose a possible common ground between evolvability of regulatory circuits, their ability to solve complex problems, and the success of overparameterized models in machine learning~\cite{Howell, Rocks}. So far, these parallels remain speculative; but the fact that simple elements can enable regulatory circuits to perform a form of associative learning could be a valuable  piece of this puzzle. In this work, we have shown that one previously proposed mechanism successfully generalizes to higher dimensions.

% In conclusion, we based on a generalized end-product feedback architecture, extended it to high dimensional case, and studied its response to changing environmental fluctuation structure.

% The results of our model show significant ‘learning’ feature of not only the overall amount fluctuation of the input environmental resources, but also the subtle fluctuation structure of it.

% First, we find that improving performance beyond SEPI is very costly. 

% However, improving learning of angular fluctuation structure can be achieved by having more regulators without extra cost. 

% Thirdly, our generalized end-product feedback architecture is sensitive to environmental change and successfully tracks the changing dominant eigenvector of fluctuation statistics $M$. 

% Finally, it also responses to change of non-dominant eigenvectors of the input fluctuation and ‘learns’ to track it. 

% [Some recent ideas propose that a cell's regulatory architecture should be seen as a high-dimensional learning circuit. / Common ground between evolvability of regulatory circuits, their ability to solve complex problems, and the success of overparameterized models in machine learning.]


\begin{thebibliography}{9}
\bibitem{Tagkopoulos}
Tagkopoulos, Ilias, Yir-Chung Liu, and Saeed Tavazoie. "Predictive behavior within microbial genetic networks." science 320.5881 (2008): 1313-1317.
\bibitem{Savageau}
M. A. Savageau, Escherichia coli habitats, cell types, and molecular mechanisms of gene control, The American Naturalist 122, 732 (1983).
\bibitem{Pedersen}
D. Bell-Pedersen, V. M. Cassone, D. J. Earnest, S. S. Golden, P. E. Hardin, T. L. Thomas, and M. J. Zo- ran, Circadian rhythms from multiple oscillators: lessons from diverse organisms, Nature Reviews Genetics 6, 544 (2005).
\bibitem{Turing}
Turing, A. M. (1936). "On Computable Numbers, with an Application to the Entscheidungsproblem" (PDF). Proceedings of the London Mathematical Society. 2. 42: 230–265. doi:10.1112/plms/s2-42.1.230. S2CID 73712.
\bibitem{Brainerd}
Brainerd, W. S.; Landweber, L. H. (1974). Theory of Computation. Wiley. ISBN 0-471-09585-0.
\bibitem{Perez}
Pérez, Jorge, Javier Marinković, and Pablo Barceló. "On the turing completeness of modern neural network architectures." arXiv preprint arXiv:1901.03429 (2019).
\bibitem{Yi}
Yi, Tau-Mu, et al. "Robust perfect adaptation in bacterial chemotaxis through integral feedback control." Proceedings of the National Academy of Sciences 97.9 (2000): 4649-4653.
\bibitem{Chang}
Chang, Hao, and Andre Levchenko. "Adaptive molecular networks controlling chemotactic migration: dynamic inputs and selection of the network architecture." Philosophical Transactions of the Royal Society B: Biological Sciences 368.1629 (2013): 20130117.
\bibitem{Husain}
K. Husain, W. Pittayakanchit, G. Pattanayak, M. J. Rust, and A. Murugan, Kalman-like self-tuned sensitivity in biophysical sensing, Cell Systems 9, 459 (2019).
\bibitem{Sorek}
Sorek, M., Balaban, N. Q., \& Loewenstein, Y. (2013). Stochasticity, bistability and the wisdom of crowds: a model for associative learning in genetic regulatory networks. PLoS computational biology, 9(8), e1003179.
\bibitem{Stefan}
Landmann, S., Holmes, C. M., \& Tikhonov, M. (2021). A simple regulatory architecture allows learning the statistical structure of a changing environment. ELife, 10. https://doi.org/10.7554/eLife.67455
\bibitem{Pavlov}
Pavlov, M. Y., \& Ehrenberg, M. (2013). Optimal control of gene expression for fast proteome adaptation to environmental change. Proceedings of the National Academy of Sciences, 110(51), 20527-20532.
\bibitem{Hart}
Hart, Y., Antebi, Y. E., Mayo, A. E., Friedman, N., \& Alon, U. (2012). Design principles of cell circuits with paradoxical components. Proceedings of the National Academy of Sciences, 109(21), 8346-8351.
\bibitem{Laughlin}
Laughlin, S. B. (2001). Energy as a constraint on the coding and processing of sensory information. Current opinion in neurobiology, 11(4), 475-480.
\bibitem{Lan}
Lan, G., Sartori, P., Neumann, S., Sourjik, V., \& Tu, Y. (2012). The energy–speed–accuracy trade-off in sensory adaptation. Nature physics, 8(5), 422-428.
\bibitem{Olson}
Olson, A. C., Patro, J. N., Urban, M., \& Kuchta, R. D. (2013). The energetic difference between synthesis of correct and incorrect base pairs accounts for highly accurate DNA replication. Journal of the american chemical society, 135(4), 1205-1208.
\bibitem{Liberzon}
Liberzon D. 2011. Calculus of Variations and Optimal Control Theory: A Concise Introduction. Princeton University Press.
\bibitem{Howell}
Howell, Owen, et al. "Machine learning as ecology." Journal of Physics A: Mathematical and Theoretical 53.33 (2020): 334001.
\bibitem{Rocks}
Rocks, Jason W., and Pankaj Mehta. "The Geometry of Over-parameterized Regression and Adversarial Perturbations." arXiv preprint arXiv:2103.14108 (2021).
\end{thebibliography}

\clearpage
\section{Supplementary Information}
\subsection{Simple end-product inhibition}
When discussing learning the average demand $\vec{\langle D\rangle}$ in the model part, we introduced the simple end-product inhibition architecture. The specific dynamics of SEPI is given by the following equations (reproduced from Ref.~\cite{Stefan}):

\begin{equation}
    \begin{aligned}
    \dot{x}=P-D\frac{x}{x_0} \ &\mathrm{source-sink\ dynamics\ of\ metabolite\ x} \\
    P = aP_0 \ &\mathrm{definition\ of\ regulator\ activity\ a} \\
    \dot{a} = \frac{x_0 - x}{\lambda} \ &\mathrm{regulator\ activity\ inhibited\ by\ x}\\
    \end{aligned}
\end{equation}

Under the assumption that the dynamics of metabolite concentrations $x$ are faster than regulatory processes, and choosing the units so that $x$ and $P$ are dimensionless, SEPI dynamics becomes~\cite{Stefan}:

\begin{equation}
    \begin{aligned}
    x&=\frac{P}{D} \\
    P&=a \\
    \tau_a\dot{a}&=1-x \\
    \end{aligned}
    \label{eq:4}
\end{equation}

It is easy to see that SEPI learns the average demand over recent past within a time scale of $\tau_a$. Indeed, derived from Eq.~\ref{eq:4}, we get:
\begin{equation}
    \tau_a\dot{a}(t)=1-\frac{a(t)}{D(t)}
\end{equation}

Which gives
\begin{equation}
    \frac{1}{D(t)-a(t)}\mathop{dD(t)}-\mathop{d\ln(D(t)-a(t))}=\frac{\mathop{dt}}{\tau_aD}
    \label{SEPI:deduction}
\end{equation}

Noticing that $\langle\dot{D}(t)\rangle=0$. Therefore on average, the first term in Eq~\ref{SEPI:deduction} cancels. Integrating it over time leads to:

\begin{equation}
    \langle a(t)\rangle=\overline{D}-e^{-\frac{t}{\tau_a\overline{D}}}
    \label{SEPI:result}
\end{equation}

Where $\overline{D}$ is the average demand. The result of Eq.~\ref{SEPI:result} confrims that SEPI learns the average demand in a time scale of $\tau_a$.

\subsection{Charge repel simulation for distributing $\sigma_{\mu i}$}

% In higher dimensional case, we choose matrix $\sigma$ to be minimally redundant. We did so by simulating rows of $\sigma$ as point charges repelling each other in a $N_x$ dimensional space, where $N_x$ is the number of metabolites.

Matrix $\sigma$ in Eq.~\ref{eq:2} represents regulatory pathways in a regulatory architecture. The $\mu$th row of it describes how the activity of regulator $a_\mu$ activates or inhibits various production $P$'s. 

We claim that regulatory pathways $\sigma_{\mu}$'s in higher dimensions need to be designed minimally redundantly. This requires $\sigma_\mu$'s to be scattered as dispersively as possible to reduce their relevance and explore the largest parameter space. As an example, if all $\sigma_\mu$'s point in the same direction, the performance will be poor. 

To achieve an overall good performance, we require $\sigma_\mu$'s to be distributed as far away from each other as possible. For example, in 1d case, the natural configuration is an activator and a repressor. In 2d, Landmann et al chose them as uniformly spaced points on a circle~\cite{Stefan}. Here, we generalize this to higher dimensions by simulating $\sigma_\mu$'s as point charges repelling each other in a $N_x$ dimensional space.

%  (1) we need to choose sigmas somehow. (2) It's clear that if they all point in the same direction (are redundant), performance will be poor. (3)  Heuristically, we want them to "be distributed as widely as possible". (4) For example, in 1d, the natural configuration is an activator + a repressor. (5) In 2d, Landmann et al chose them as uniformly spaced points on a circle. (6) We generalize this to higher dimensions by considering repelling charges on a sphere. (7) Provide details.



\end{document}