Dynamical Parameters: An Interpretability Framework for Time-Series Foundation Models

Dynamical Laws and Parameters. Each time series is defined by a dynamical law $m\in\left\{1,\ldots,M\right\}$ and a parameter vector that shapes its temporal evolution

The law $m$ specifies the form of the dynamics, while $z_{m}$ contains the quantitative properties governing the evolution of the series. Each component $z_{m,j}$ is a Dynamical Parameter. Every law instantiates a common state-space form in which $z_{m}$ controls the transition dynamics, input, or readout.

Given $m$, $z_{m}$, and stochastic variation $\varepsilon$, the controlled generator produces a history and future:

The TSFM observes only the history and produces a forecast and layer-wise hidden representations:

The generator exposes the law and parameters that the TSFM can recover only from the history.

Parameter Interventions. To isolate the effect of parameter $z_{m,j}$, two parameter vectors are constructed that differ only in that component:

Both sequences are generated using the same stochastic realization $\varepsilon$:

Here, $x^{A}$ and $x^{B}$ are the matched histories, while $y^{A}$ and $y^{B}$ are their futures. Sharing $\varepsilon$ removes stochastic variation unrelated to the parameter intervention. The resulting reference response is

The known law $m$, Dynamical Parameter vector $z_{m}$, and reference response $\Delta y^{\mathrm{ref}}_{m,j}$ provide targets for analyzing representation accessibility, forecast response, and causal geometry.

Representation accessibility measures how accurately the dynamical law $m$ and its parameters $z_{m}$ can be recovered from hidden representations using linear probes. The analysis has three steps.

(I) Probing Laws and Parameters. At each layer $\ell$, a linear classifier predicts the dynamical law from $h^{\ell}$, while a linear ridge probe predicts each parameter within its corresponding law:

(II) Dynamical Law Decodability. For each law $m\in\left\{1,\ldots,M\right\}$, let $A_{m}^{\star}$ denote an upper reference for identifying $m$. For balanced $M$-way classification, $A^{0}=1/M$ is the uniform-guessing accuracy. The layer-wise accuracy and normalized law decodability are

When a reliable Bayes ceiling is unavailable, $A_{m}^{\star}=1$ provides a conservative upper reference. A value of $D_{m}\left(\ell\right)$ near $1$ indicates that the hidden representation approaches this reference, while a value near $0$ indicates chance-level identification.

(III) Parameter Accessibility. For each Dynamical Parameter $z_{m,j}$, we compare the probe correlation $r_{m,j}^{\ell}$ with a conditional posterior-mean reference:

Here $\eta$ collects the scale, offset, noise, phase, and non-designated parameters supplied to the reference observer but not to the TSFM. A value of $D_{m,j}(\ell)$ near $0$ indicates no linear association, while a value near $1$ means the probe approaches this law- and nuisance-informed reference.

Forecast response measures whether changing a Dynamical Parameter changes the TSFM forecast in the direction, magnitude, and temporal shape implied by the law. The analysis has three steps.

(I) Matched Forecast Response. For the matched pair in Equation 2, the TSFM forecast response is

(II) Response Agreement. To compare the forecast and reference responses while allowing a constant level offset, define the centered horizon response

For $N$ matched pairs, the per-pair response coefficient and its mean are

Here, $u_{i,m,j}$ is the least-squares slope of the forecast response on the reference response, with a free level offset. The mean $U_{m,j}$ near $1$ indicates a reference-aligned component of the expected size, near $0$ indicates little aligned response, and a negative value indicates an opposing component.

Full-Vector Fidelity and Aggregation. The response coefficient $U_{m,j}$ measures only the component aligned with the reference and therefore does not establish fidelity of the full forecast-change trajectory. For each matched pair $i$, let $q_{i}$ denote the realized forecast change and $r_{i}$ the corresponding reference change. Define $\widetilde{q}_{i}=\mathcal{C}(q_{i})$, $\widetilde{r}_{i}=\mathcal{C}(r_{i})$, $E=\sum_{i}\|\widetilde{r}_{i}\|_{2}^{2}$, and the pairwise aligned gain $g_{i}=\langle\widetilde{q}_{i},\widetilde{r}_{i}\rangle/\|\widetilde{r}_{i}\|_{2}^{2}$. We report four pooled metrics:

Here, $g$ measures the reference-aligned component, $s$ measures directional agreement, $e$ measures full centered-trajectory error, and $e_{\perp}$ measures the component orthogonal to each pair’s reference. These quantities satisfy the pooled decomposition $e^{2}=\sum_{i}\|\widetilde{r}_{i}\|_{2}^{2}(g_{i}-1)^{2}/E+e_{\perp}^{2}$; thus, total error reflects both gain mismatch and orthogonal trajectory error. For the natural response, $g_{i}=u_{i,m,j}$, so $U_{m,j}=N^{-1}\sum_{i}g_{i}$ weights pairs equally, whereas pooled $g$ weights each pair by its reference energy. We report both because a few large reference changes can dominate the pooled metric.

(III) Conditional-Reference-Normalized Response. Finite histories and stochastic variation can prevent even an optimal history-based forecaster from reproducing the realized reference response. We therefore define the law- and nuisance-informed conditional forecast

where $\eta$ collects the documented nuisance variables supplied to the conditional forecast. Substituting $\Delta y_{i,m,j}^{\star}$ for $\Delta\hat{y}_{i,m,j}$ in Equation 7 gives the conditional-reference coefficient $U_{m,j}^{\star}$. Across intervention magnitudes $\mathcal{M}_{m,j}$, the normalized forecast response is

A value of $\bar{U}_{m,j}$ near $1$ means the model matches the conditional reference’s aggregate aligned component across magnitudes; near $0$ means little alignment, and negative values mean an opposing response. Because this signed normalization conditions on $m$ and $\eta$, it is a comparison, not a history-only upper bound: $\bar{U}_{m,j}$ can exceed one and does not establish full-vector fidelity.

Causal geometry characterizes how hidden-state interventions affect the frozen downstream computation and whether their effective directions share structure across inputs. Here, “causal” refers only to the effect of modifying the hidden state at a specified layer; it neither identifies causal variables in the generating process nor establishes that the model naturally uses the resulting directions. For a fixed law $m$, parameter $z_{m,j}$, and intervention magnitude $\delta$, we suppress dependence on $m$, $j$, and $\delta$ below.

(I) Local Causal Directions. For matched pair $i$ at layer $\ell$, let $h_{i}^{\ell}=F_{\theta}^{\ell}(x_{i}^{A})$ denote the representation of $x_{i}^{A}$, and let $F_{\theta}^{>\ell}$ denote the remaining computation to the forecast. The intervention modifies the selected hidden-state positions while leaving the downstream computation fixed. Linearizing the forecast around $h_{i}^{\ell}$, regularized Jacobian inversion yields

Here, $J_{i}^{\ell}$ maps hidden-state perturbations to first-order forecast changes, and $v_{i}^{\ell}$ is the regularized, input-specific local causal direction. We evaluate this direction in the nonlinear TSFM and summarize agreement across $N$ matched pairs. For intervention strength $\alpha>0$, the realized response per unit intervention and pooled aligned gain are

The inversion targets the complete reference response. The primary aligned-gain score removes constant level offsets using $\mathcal{C}$ in Equation 6, whereas raw and level diagnostics retain them. A value of $U^{\mathrm{pool}}$ near $1$ indicates the correct pooled aligned component, near $0$ indicates little aligned response, and a negative value indicates an opposing component. Setting $v_{i}=v_{i}^{\ell}$ defines $U_{\mathrm{local}}(\ell)$, which measures whether the Jacobian-derived directions remain effective in the nonlinear model.

(II) Shared Causal Subspace. Although the local causal directions are input-specific, they may concentrate in a compact subspace shared across inputs. The subspace is estimated from a discovery bank $\mathcal{D}_{\mathrm{disc}}$ and evaluated on a disjoint bank $\mathcal{D}_{\mathrm{eval}}$. At layer $\ell$, singular-value decomposition gives

where $Q_{\ell,k}$ contains the leading $k$ left singular vectors. Substituting the projected directions $\tilde{v}_{i}^{\ell,k}$ into Equation 11 on $\mathcal{D}_{\mathrm{eval}}$ defines $U_{\mathrm{projected}}\left(\ell,k\right)$.

For each valid layer, the smallest rank retaining $90\%$ of the local reference-aligned gain is

A small $k_{90}(\ell)$ indicates a low-rank shared causal subspace relative to the intervened state dimension. If no evaluated rank reaches the threshold, $k_{90}\left(\ell\right)$ is reported as a lower bound. However, $\tilde{v}_{i}^{\ell,k}$ still requires the reference-derived local direction $v_{i}^{\ell}$, so this analysis establishes subspace compactness but not whether its input-specific coordinates are accessible from the hidden representation.

(III) Causal Coordinate Accessibility. The final step asks whether the input-specific coordinates of the local causal direction can be inferred from the unperturbed representation $h_{i}^{\ell}$. The local direction is projected into the shared subspace, its coordinates are predicted from $h_{i}^{\ell}$, and the predicted direction is mapped back into the hidden space:

Here, $P_{\ell,k}$ and $b_{\ell,k}^{c}$ are the ridge-readout coefficients in the original hidden coordinates and incorporate discovery-bank standardization. The readout alone is fitted on $\mathcal{D}_{\mathrm{disc}}$, while the TSFM remains frozen. On $\mathcal{D}_{\mathrm{eval}}$, the predicted direction $\hat{v}_{i}^{\ell,k}$ requires neither $J_{i}^{\ell}$ nor $\Delta y_{i}^{\mathrm{ref}}$. Using these directions in Equation 11 defines the deployed response score $U_{\mathrm{deployed}}(\ell,k)$.

A fixed-direction control uses the evaluation-bank mean direction, rescaled to match each input’s oracle norm. Its response score is $U_{\mathrm{fixed}}\left(\ell\right)$. Because the control receives the oracle norm, comparison with $U_{\mathrm{deployed}}\left(\ell,k\right)$ isolates the benefit of input-specific coordinates from perturbation magnitude. High projected response at low rank indicates a compact shared subspace, while high deployed response shows that its input-specific coordinates are accessible from unperturbed representations.

Dynamical-law identification tests whether TSFM representations distinguish the thirteen laws. Balanced training, validation, and test banks treat each multi-parameter law as one class with jointly varying parameters. A multinomial logistic probe is fitted at each layer, with layer and penalty selected on validation data. For $D_{m}(\ell)$, we set chance accuracy to $A_{0}=1/13$ and the reference to $A_{m}^{\star}=1$, yielding normalized above-chance accuracy. Results appear in Figure 1.

Dynamical Laws Are Broadly Decodable. Every model distinguishes the thirteen laws far above chance. Mean law decodability ranges from $0.757$ to $0.971$ across models, with a mean of $0.929$. Thus, law identity remains broadly accessible even when parameters vary jointly, although this does not establish a unique learned mechanism.

Difficulty Concentrates in Related Dynamics. Forced diffusion (FD) has the lowest average decodability of $0.842$ and is the weakest law for seven models. Random walk with drift (RW) averages $0.851$ and is weakest for Moirai (MO) and TTM-R2 (TT). Other laws average at least $0.877$, indicating that errors concentrate in a small subset of dynamical families.

Differences Are Model- and Law-Specific. Seven models average between $0.939$ and $0.971$, whereas Moirai (MO) averages $0.908$ and TTM-R2 (TT) averages $0.757$. TTM-R2 (TT) scores $0.51$ on random walk with drift (RW) and $0.56$ on forced diffusion (FD). They are its weakest laws. TimesFM 3 (F3) scores $0.92$ on seasonal autoregression (SA), compared with $0.99$ for TimesFM 2.5 (F2), showing that aggregate scores can obscure law-specific differences. Complete layer-wise profiles and confusion analyses for all models and thirteen laws appear in §B.1.

We study ten single-parameter laws. Accessibility covers eight, excluding random walk with drift (RW) and level shift (LS) because their required references are unavailable. Forecast response and causal geometry cover seven, additionally excluding exponential relaxation (RX) because its response reference is unstable. The common comparison spans seven laws and nine models, totaling 63 cells.

For seven parameters across three multi-parameter laws, each intervention changes one parameter while holding the others and the stochastic realization fixed. We compare representation accessibility, matched-input forecast response, and deployed aligned gain. Figure 6 summarizes the three measurements.

Joint Parameters Remain Accessible. Median accessibility is $0.977$. For linear state space, modal frequency (SC) exceeds eigenvalue modulus (SR) by $0.08$–$0.30$ in every model. Adding a matched high tone (TH) reduces low-tone (TL) accessibility by at most $0.033$ in six models. Joint components therefore produce limited interference, but accessibility still varies across parameters of the same law.

Accessibility and Forecast Response Diverge. Drive frequency (WC) is accessible at $0.98$–$1.00$, yet no matched-input response exceeds $0.56$. Diffusion time (WD) is less accessible, but six models attain responses between $0.76$ and $0.96$. This reversal within forced diffusion further separates accessibility from forecast response under jointly varying parameter settings.

Predicted Edits Partially Recover the Response. Deployed directions outperform the fixed control in 56 of 63 cells across all three multi-parameter laws. Median centered errors are $0.93$, $0.14$, and $0.61$ for matched-input, projected, and deployed responses. TTM-R2 (TT), Timer-XL (TX), and Moirai (MO) attain deployed gains between $0.16$ and $0.73$ despite weak or opposing matched-input responses. Thus, accessible parameters can support the reference response without being fully expressed by input changes or predicted edits. Further analyses appear in §B.5.

The controlled-law experiments isolate parameter effects but use synthetic backgrounds. To test whether the accessibility–response separation persists on real data, centered trend or sinusoid components are injected into ETTh1 Zhou et al. (2021) windows. Matched pairs share the background and differ only in the injected component. Injection strength is the component root-mean-square (RMS) amplitude relative to context standard deviation, at $0.25$, $0.5$, and $1.0$. The analytic future difference defines the reference response. All nine TSFMs are evaluated on $2{,}048$ test windows per setting. No-injection and shuffled-future controls yield median pooled gains of $0.000$ and $-0.000$.

Results. Across the evaluation, median probe correlation remains high at $0.94$, while pooled matched-input gain is $-0.50$ for linear trend and $0.36$ for sinusoid. As shown in Figure 7, ETTh1 exhibits a separation between representation accessibility and forecast response across three injection strengths, with linear trend gain remaining negative from $-1.40$ at strength $0.25$ to $-0.62$ at strength $1.0$, while sinusoid gain increases from $0.09$ to $0.67$. The response depends on the background: at strength $1.0$, linear trend gain reaches $0.89$ on Electricity and $0.71$ on Weather, while Weather sinusoid gain reaches $0.77$. Thus, the accessibility–response separation persists on real-series backgrounds, although its magnitude and sign vary with the background and injection strength. These gains use the injected continuation rather than a conditional reference for the background. The full comparison across ETTh1, Electricity, and Weather is reported in §B.6.

Additional Analyses. The appendix reports mechanistic diagnostics in §B.7, reference sensitivity in §B.8, intervention side effects in §B.9, computational cost in §B.10, and limitations in §B.11. Together, these analyses characterize the robustness and scope of the main findings.