A Simplex Witness Certificate and Escape Force for Constant Collapse in Variational Autoencoders

Extracted source text · arXiv 2605.18224

\documentclass[11pt]{article}
\usepackage[preprint,nonatbib]{neurips_2026}
\usepackage{amsmath,amssymb,amsfonts,amsthm,mathtools}
\usepackage{microtype}
\usepackage{booktabs}
\usepackage{enumitem}
\usepackage{hyperref}
\usepackage{graphicx}
\usepackage{float}
\usepackage{tikz}
\usetikzlibrary{arrows.meta,positioning,calc}

\hypersetup{colorlinks=true,linkcolor=blue,citecolor=blue,urlcolor=blue}

\title{A Simplex Witness Certificate and Escape Force for Constant Collapse in Variational Autoencoders}

\author{
  Zegu Zhang\textsuperscript{1} \qquad Jianhua Peng\textsuperscript{2} \qquad Jian Zhang\textsuperscript{3}\\
  \texttt{\{zeguzhang@outlook.com, 2018100913@niit.edu.cn, zhangjian.1993@tsinghua.org.cn\}}\\
  \textsuperscript{1}Independent Researcher \quad \textsuperscript{2}School of Computing, Southeast University \quad \textsuperscript{3}Independent Researcher
}
\date{}

\newtheorem{definition}{Definition}
\newtheorem{lemma}{Lemma}
\newtheorem{theorem}{Theorem}
\newtheorem{proposition}{Proposition}
\newtheorem{corollary}{Corollary}
\newtheorem{remark}{Remark}

\newcommand{\E}{\mathbb{E}}
\newcommand{\R}{\mathbb{R}}
\newcommand{\N}{\mathcal N}
\newcommand{\KL}{\mathrm{KL}}
\newcommand{\softmax}{\operatorname{softmax}}
\newcommand{\LTS}{L_{\mathrm{TS}}}
\newcommand{\IT}{I_T}
\newcommand{\barT}{\bar T}
\newcommand{\ET}{\mathcal E_T}
\newcommand{\G}{G_T}

\begin{document}
\maketitle

\begin{abstract}
We study exact constant collapse in variational autoencoders: the deterministic encoder mean becomes independent of the input. The prior remains the standard Gaussian. Before VAE training, we select a fixed teacher posterior from a GMM-based view of the data and attach a fixed latent-only simplex witness to the encoder mean. This construction yields two linked objects. The first is a certificate: if the witness prediction improves on the best constant predictor of the teacher, the encoder mean cannot be input-independent constant. The second is a local escape direction: on the collapsed manifold, the teacher residual gives a sample-dependent descent direction for the alignment loss. For any full-support teacher posterior, the same geometry also gives a closed-form latent code with zero teacher-witness alignment error. Its scaled versions trace a margin-energy path from the constant predictor to the exact teacher code, which quantifies non-collapse inside the protected witness subspace. We instantiate the method on MNIST, CIFAR-10, and CIFAR-100. With searched unsupervised PCA-GMM teachers, vanilla VAEs fail the teacher-witness certificate in all five seeds on CIFAR-10 and CIFAR-100, while RST variants pass in all five seeds. Under collapse-stress settings with $\beta_{\mathrm{KL}}\in\{2,4,8\}$, vanilla VAE again fails in all seeds, whereas RST-alpha-prefit remains certificate-positive. Escape trajectories on both natural-image datasets increase the witness margin from a low-margin initialization and exhibit nonzero teacher-induced gradient norms. The analysis is confined to exact constant collapse of the encoder mean; generation quality, decoder use, and other collapse modes remain separate questions.
\end{abstract}

\section{Introduction}

Posterior collapse is usually diagnosed only after training: one trains a VAE and then asks whether the latent variable has carried input-dependent information. That workflow is inadequate when a particular collapse mode is to be excluded by design. This work isolates one such mode, exact input-independent constant collapse of the encoder mean.

We keep the prior fixed as $p(z)=\mathcal N(0,I)$ and introduce a single teacher posterior over the data. The teacher is chosen before VAE training and then held fixed. It enters neither the decoder nor the prior; it defines a target view against which the latent mean is tested.

The mechanism is a teacher-student witness path. A searched GMM teacher gives a posterior vector $T_x$ for each data point. A fixed simplex witness maps the encoder mean $\mu_\phi(x)$ to $S(\mu_\phi(x))$. The loss $L_{\mathrm{TS}}=\mathbb E_x\mathrm{KL}(T_x\|S(\mu_\phi(x)))$ compares these two distributions. Since the witness reads only the latent mean, a constant encoder mean produces a constant witness prediction. Beating the best constant predictor of the teacher therefore certifies that the encoder mean is not input-independent constant.

The same loss also describes a local escape mechanism. At exact collapse, the negative alignment gradient contains the teacher residual $T_x-\bar T$. Thus the collapsed manifold is not an absorbing set for the witness dynamics when the teacher is nontrivial and the encoder has a tangent direction that can express the induced signal. This distinguishes RST from a purely post-hoc certificate.

This work builds on our earlier teacher-guided VAE certificate, which introduced a testable constant-predictor baseline and used an inequality of the form $L<I_T$ as a final non-collapse certificate~\cite{zhang2026teacher}. Here the teacher has an additional role: it defines a fixed latent-only witness whose alignment loss supplies the transverse escape direction. The resulting framework treats feasibility, escape, and final certification as parts of the same construction.

The fixed witness has an explicit inverse on the teacher view. For a full-support teacher posterior, centered teacher log-odds embed into the standard-Gaussian latent space so that the witness exactly recovers the teacher. This gives a closed-form teacher code and a computable latent mean energy. Scaled codes along the ray from the prior center to this analytic target provide a margin-energy tradeoff; see Appendix~\ref{app:alpha-target}. Teacher search is therefore guided by three quantities: teacher information, log-odds energy, and stability.

The scaled path also clarifies the meaning of partial non-collapse. It is not a coordinate-wise active-unit statement. Instead, it measures how much teacher information is expressed in the witness subspace: $\alpha=0$ gives the best constant predictor, while $\alpha=1$ gives exact teacher matching. Coarsened or hierarchical teacher views can be tested with their own constant baselines, yielding certificates at progressively finer levels of the teacher view without asserting full-dimensional latent usage.

\paragraph{Contributions.}
\begin{enumerate}[leftmargin=1.5em]
    \item We give a constant-collapse certificate based on one searched teacher posterior and one fixed latent-only simplex witness.
    \item We prove a function-space escape result: at exact collapse, the witness alignment loss has a nonzero sample-dependent descent direction whenever the teacher is nontrivial.
    \item We derive a closed-form latent target with zero teacher-witness alignment error, together with its latent mean energy and its scaled margin-prior tradeoff.
    \item We state the certificate's scope precisely: scaled targets quantify the margin-energy tradeoff inside the teacher-witness subspace, not full-dimensional latent usage.
\end{enumerate}

\paragraph{Empirical protocol.}
We evaluate the construction in five parts: a controlled MNIST sanity check, searched-teacher feasibility on CIFAR-10 and CIFAR-100, five-seed training certificates, collapse-stress runs, and escape trajectories. The appendices record the supporting constructions: simplex geometry, teacher search, scaled targets, witness variants, prior-matching diagnostics, and optional reconstruction-side measurements.

\paragraph{Scope of the result.}
The certificate concerns exact constant collapse of the deterministic encoder path. The main theorem uses only the teacher-student witness path: the teacher gives $T_x$, the witness reads $\mu_\phi(x)$, and the margin checks whether this prediction improves on the best constant predictor. Reconstruction-side teacher evaluations, described in Appendix~\ref{app:reconstruction-side}, are diagnostics rather than assumptions of the theorem. The result does not replace measurements of generation quality, decoder use, per-dimension KL, active units, or other representation metrics.

\section{Setup and notation}
\label{sec:setup}

\subsection{Standard-Gaussian VAE}

Let $x\sim P_X$. The VAE encoder is
\[
q_\phi(z\mid x)=\N(\mu_\phi(x),\operatorname{diag}\sigma_\phi^2(x)),
\]
and the prior is fixed as
\[
p(z)=\N(0,I).
\]
The decoder is denoted by $p_\theta(x\mid z)$ or $\hat x=D_\theta(z)$. The witness acts on the encoder mean $\mu_\phi(x)$.

\subsection{Searched GMM teacher}
\label{sec:gmm-teacher}

Before VAE training, a fixed teacher posterior is selected. Let $\psi_\omega(x)\in\R^m$ be a candidate representation. For a search choice $\omega$, fit a $K$-component GMM
\[
p_\omega(u)=\sum_{k=1}^K \pi_k\N(u;m_k,\Sigma_k),\qquad u=\psi_\omega(x).
\]
The GMM responsibility is
\[
\gamma_\omega(k\mid x)=
\frac{\pi_k\N(\psi_\omega(x);m_k,\Sigma_k)}
{\sum_{j=1}^K\pi_j\N(\psi_\omega(x);m_j,\Sigma_j)}.
\]
After temperature and probability smoothing, the selected teacher posterior is written as
\[
T_x(k)=T_{\omega^\star}(c=k\mid x)\in\Delta_K.
\]
Here
\[
\Delta_K=\left\{p\in\R_+^K:\sum_{k=1}^K p_k=1\right\}
\]
denotes the probability simplex. The teacher posterior is fixed before VAE training.

Define
\[
\bar T=\E_xT_x,
\qquad
I_T=\E_x\KL(T_x\|\bar T).
\]
The teacher is nontrivial when $I_T>0$.

\subsection{Centered log-odds and teacher energy}

For a full-support teacher, define
\[
r_k(x)=
\log\frac{T_{x,k}}{\bar T_k}
-
\frac1K\sum_{j=1}^K\log\frac{T_{x,j}}{\bar T_j}.
\]
Then $\sum_k r_k(x)=0$. The teacher log-odds energy is
\[
\mathcal E_T=\E_x\sum_{k=1}^K r_k(x)^2.
\]
It measures the latent mean energy needed to encode the teacher posterior through the fixed witness.

\subsection{Fixed simplex witness}

Assume $d_z\ge K-1$. Let $v_1,\ldots,v_K\in\R^{d_z}$ be regular simplex vertices satisfying
\[
\sum_{k=1}^K v_k=0,
\qquad
v_i^\top v_j=
\begin{cases}
1,& i=j,\\
-\frac1{K-1},& i\ne j.
\end{cases}
\]
For a fixed gain $\beta>0$, define
\[
S_k(z)=\softmax_k(\beta v_k^\top z+\log\bar T_k).
\]
The teacher-student alignment loss is
\[
L_{\mathrm{TS}}=\E_x\KL(T_x\|S(\mu_\phi(x))).
\]
The certificate margin is
\[
G_T=I_T-L_{\mathrm{TS}}.
\]
The quantities $T_x$, $\bar T$, $v_k$, $\beta$, and $S$ are fixed before optimization. The alignment term acts on the encoder through the witness output $S(\mu_\phi(x))$.

\section{Architecture}

Figure~\ref{fig:architecture} shows the data path and the witness path. The teacher defines the target $T_x$. The encoder mean is passed through the fixed witness $S$, and $L_{\mathrm{TS}}$ compares $T_x$ with $S(\mu_\phi(x))$.



\begin{figure}[H]
\centering
\resizebox{0.98\linewidth}{!}{%
\begin{tikzpicture}[>=Latex, font=\scriptsize]
\tikzstyle{box}=[draw,rounded corners,align=center,minimum height=0.72cm,inner sep=4pt]
\tikzstyle{lossbox}=[draw,rounded corners,align=center,minimum height=0.72cm,inner sep=4pt,thick]
\tikzstyle{optbox}=[draw,dashed,rounded corners,align=center,minimum height=0.72cm,inner sep=4pt]

% Standard VAE path
\node[box, minimum width=1.15cm] (x)   at (0,0)  {Input\\$x$};
\node[box, minimum width=1.55cm] (enc) at (2.45,0){Encoder\\$q_\phi(z\mid x)$};
\node[box, minimum width=1.45cm] (z)   at (4.90,0){Latent\\$\mu_\phi(x),z$};
\node[box, minimum width=1.65cm] (dec) at (7.35,0){Decoder\\$p_\theta(x\mid z)$};
\node[box, minimum width=1.20cm] (rec) at (9.85,0){Recon.\\$\hat x$};
\draw[->] (x) -- (enc);
\draw[->] (enc) -- (z);
\draw[->] (z) -- (dec);
\draw[->] (dec) -- (rec);

% Main witness and teacher layers
\node[box, minimum width=2.75cm] (witness) at (4.90,1.50){Fixed witness\\$S(z)=\softmax(\beta Vz+\log\bar T)$};
\node[box, minimum width=2.25cm] (teach) at (0,3.00){Searched teacher\\$T_x=T_{\omega^\star}(c\mid x)$};
\node[lossbox, minimum width=3.10cm] (align) at (4.90,3.00){Alignment\\$L_{\mathrm{TS}}=\KL(T_x\Vert S(\mu_\phi(x)))$};
\node[box, minimum width=2.10cm] (stats) at (0,4.50){Precomputed\\$\bar T,I_T,\mathcal E_T$};
\node[lossbox, minimum width=2.20cm] (cert) at (8.10,4.50){Certificate\\$G_T=I_T-L_{\mathrm{TS}}$};

\draw[->] (x.north) -- (teach.south);
\draw[->] (teach.north) -- (stats.south);
\draw[->] (z.north) -- (witness.south);
\draw[->] (witness.north) -- (align.south);
\draw[->] (teach.east) -- (align.west);
\draw[->] (stats.east) -- (cert.west);
\draw[->] (align.north east) -- (cert.south west);

% Optional reconstruction-side diagnostics
\node[optbox, minimum width=2.15cm] (rhat) at (10.25,1.50){Optional\\$R_{\hat x}=T_{\omega^\star}(c\mid\hat x)$};
\node[optbox, minimum width=2.35cm] (optloss) at (10.25,3.00){Diagnostics\\$L_{TR},\ L_{SR}$};
\draw[dashed,->] (rec.north) -- (rhat.south);
\draw[dashed,->] (rhat.north) -- (optloss.south);
\draw[dashed,->] (align.east) -- (optloss.west);

% VAE objective
\node[lossbox, minimum width=2.45cm] (vae) at (4.90,-1.55){VAE loss\\reconstruction $+$ KL};
\draw[->] (x.south) |- (vae.west);
\draw[->] (rec.south) |- (vae.east);
\end{tikzpicture}}
\caption{Architecture of RST. The searched teacher defines $T_x$, while the encoder mean is mapped by the fixed witness to $S(\mu_\phi(x))$. The solid alignment path gives the certificate margin $G_T=I_T-L_{\mathrm{TS}}$. Dashed reconstruction-side quantities involving $R_{\hat x}=T_{\omega^\star}(c\mid\hat x)$ are optional diagnostics described in Appendix~\ref{app:reconstruction-side}.}
\label{fig:architecture}
\end{figure}

\section{Theory}

\subsection{The constant baseline}

\begin{lemma}[Constant-predictor baseline]
For any constant predictor $\alpha\in\Delta_K$,
\[
\E_x\KL(T_x\|\alpha)=I_T+\KL(\bar T\|\alpha).
\]
Consequently,
\[
\inf_{\alpha\in\Delta_K}\E_x\KL(T_x\|\alpha)=I_T,
\]
and the minimizer is $\alpha=\bar T$.
\end{lemma}

\begin{proof}
\begin{align*}
\E_x\KL(T_x\|\alpha)
&=\E_x\sum_k T_{x,k}\log\frac{T_{x,k}}{\alpha_k}\\
&=\E_x\sum_kT_{x,k}\log\frac{T_{x,k}}{\bar T_k}
+\sum_k\bar T_k\log\frac{\bar T_k}{\alpha_k}\\
&=I_T+\KL(\bar T\|\alpha).
\end{align*}
\end{proof}

\subsection{Final certificate}

\begin{theorem}[Direct constant-collapse certificate]
If
\[
L_{\mathrm{TS}}<I_T,
\]
then $\mu_\phi(x)$ is not input-independent constant.
\end{theorem}

\begin{proof}
If $\mu_\phi(x)\equiv z_0$, then $S(\mu_\phi(x))\equiv\alpha$ is a constant predictor. The constant baseline lemma gives $L_{\mathrm{TS}}\ge I_T$, contradicting the assumption.
\end{proof}

This theorem is the final certificate: after training, $G_T=I_T-L_{\mathrm{TS}}>0$ rules out exact input-independent constant collapse.

\subsection{Function-space escape force}

The certificate above is static. The same loss also supplies a descent direction at exact collapse. Let
\[
a_k(x)=\beta v_k^\top\mu_\phi(x)+\log\bar T_k.
\]
For the alignment loss, the derivative with respect to logits is
\[
\nabla_{a(x)}\KL(T_x\|S(\mu_\phi(x)))=S(\mu_\phi(x))-T_x.
\]
At $\mu_\phi(x)\equiv0$, we have $S(0)=\bar T$. Therefore the negative logit gradient is $T_x-\bar T$, and the negative gradient with respect to the encoder mean is
\[
-\nabla_{\mu(x)}\KL(T_x\|S(\mu(x)))
=
\beta\sum_{k=1}^K(T_{x,k}-\bar T_k)v_k.
\]

\begin{proposition}[Escape at the collapsed manifold]
If $I_T>0$, then the vector field
\[
F_T(x)=\beta\sum_{k=1}^K(T_{x,k}-\bar T_k)v_k
\]
is not identically zero. Hence exact constant collapse is not a function-space stationary point of the witness alignment loss.
\end{proposition}

\begin{proof}
If $F_T(x)=0$ almost everywhere, affine independence of the simplex vertices implies $T_x=\bar T$ almost everywhere, which gives $I_T=0$. The contrapositive proves the claim.
\end{proof}

For the total objective, the RST term removes exact collapse as an absorbing state when its transverse component is not cancelled by the reconstruction and KL terms. The relevant quantities are
\[
g_{\mathrm{TS}}=\nabla_\phi L_{\mathrm{TS}},\qquad
g_{\mathrm{VAE}}=\nabla_\phi(\mathcal L_{\mathrm{rec}}+\alpha_{\mathrm{KL}}\mathcal L_{\mathrm{KL}}).
\]
The full alignment contribution is $\lambda_{\mathrm{TS}}g_{\mathrm{TS}}$. If cancellation occurs at one value of $\lambda_{\mathrm{TS}}$ and $g_{\mathrm{TS}}\ne0$, changing the alignment weight breaks that exact cancellation. Practical escape is therefore governed by a gradient-dominance condition controlled by the teacher signal, the encoder tangent response, conditioning, learning rate, and $\lambda_{\mathrm{TS}}$. The final certified statement remains $G_T>0$; optimization behavior is checked with escape trajectories, $\lambda_{\mathrm{TS}}$ sweeps, and gradient-dominance diagnostics.

\subsection{Closed-form teacher code}

\begin{lemma}[Simplex log-odds inverse]
For full-support $T_x$, define
\[
z_T(x)=\frac{K-1}{K\beta}\sum_{k=1}^K r_k(x)v_k.
\]
Then
\[
S(z_T(x))=T_x.
\]
In particular, the analytic code gives $L_{\mathrm{TS}}=0$.
\end{lemma}

\begin{proof}
Since $\sum_kr_k(x)=0$ and the $v_k$ form a regular simplex,
\[
v_i^\top\sum_kr_kv_k=\frac K{K-1}r_i.
\]
Thus $\beta v_i^\top z_T(x)=r_i(x)$. The witness logits equal
\[
r_i(x)+\log\bar T_i
=\log T_{x,i}-\frac1K\sum_j\log\frac{T_{x,j}}{\bar T_j},
\]
which differ from $\log T_{x,i}$ by a constant independent of $i$. Softmax removes this constant.
\end{proof}

\begin{lemma}[Latent energy]
The analytic code satisfies
\[
\frac12\E_x\|z_T(x)\|^2
=
\frac{K-1}{2K\beta^2}\mathcal E_T.
\]
\end{lemma}

\begin{proof}
Using $\sum_kr_k=0$ and the simplex inner products,
\[
\left\|\sum_kr_kv_k\right\|^2=
\frac K{K-1}\sum_kr_k^2.
\]
Substitution gives the result.
\end{proof}

The analytic target proves feasibility under the fixed witness: with a standard-Gaussian prior, there exists a nonconstant teacher-aligned mean code whose energy can be computed. This statement concerns the encoder mean and does not require the aggregate encoder distribution to be standard Gaussian.

\subsection{Energy comparison}

The full VAE objective is nonconvex and includes reconstruction, decoder parameters, posterior variances, and stochastic sampling. For design purposes, isolate the two terms that act directly on the encoder mean:
\[
J(\mu)=\frac{\alpha_{\mathrm{KL}}}{2}\E_x\|\mu(x)\|^2+
\lambda_{\mathrm{TS}}L_{\mathrm{TS}}.
\]
For $\mu(x)\equiv0$, the witness output is $\bar T$ and
\[
J_{\mathrm{collapse}}=\lambda_{\mathrm{TS}}I_T.
\]
For $\mu(x)=z_T(x)$, $L_{\mathrm{TS}}=0$ and
\[
J_{\mathrm{teacher}}=
\alpha_{\mathrm{KL}}\frac{K-1}{2K\beta^2}\mathcal E_T.
\]
Thus, if
\[
\alpha_{\mathrm{KL}}\frac{K-1}{2K\beta^2}\mathcal E_T
<
\lambda_{\mathrm{TS}}I_T,
\]
the analytic noncollapsed code is preferred to the exact collapsed code by this simplified mean objective.


This comparison guides teacher search and hyperparameter selection; it is separate from stochastic-gradient convergence.

\subsection{Scaled targets, group views, and partial collapse}
\label{sec:alpha-partial}

The analytic target also gives a simple way to quantify controlled departure from constant collapse. For
\[
\mu_\alpha(x)=\alpha z_T(x),\qquad 0\le \alpha\le 1,
\]
the witness prediction follows the exponential interpolation
\[
S_k(\mu_\alpha(x))
=
\frac{T_{x,k}^{\alpha}\bar T_k^{1-\alpha}}
{\sum_j T_{x,j}^{\alpha}\bar T_j^{1-\alpha}} .
\]
Hence $L_{\mathrm{TS}}(0)=I_T$ and $L_{\mathrm{TS}}(1)=0$. Along this ray, increasing $\alpha$ moves the code continuously from the best constant predictor to exact teacher matching, while the mean energy grows quadratically:
\[
\frac12\E_x\|\mu_\alpha(x)\|^2
=
\alpha^2\frac{K-1}{2K\beta^2}\mathcal E_T .
\]
Thus $\alpha$ is an interpretable design knob: it controls how much teacher information is written into the witness subspace per unit of latent energy.

This observation is weaker than full dimensional latent usage. The witness only sees the simplex subspace
\[
\mathcal U=\operatorname{span}\{v_1,\ldots,v_K\},
\]
whose dimension is at most $K-1$. A small $L_{\mathrm{TS}}$ certifies nonconstant behavior in the teacher-witness view. It does not certify that every coordinate, every posterior variance, or every direction in $\mathcal U^\perp$ carries input information.

Coarser teacher views can be analyzed by applying the same constant-baseline argument after merging teacher components. Such views are useful for interpreting partial non-collapse: a coarse teacher may cross its own constant baseline before the full $K$-component teacher does. The main theorem, however, remains the single-teacher certificate based on $G_T=I_T-L_{\mathrm{TS}}$. Appendix~\ref{app:group-hierarchy} gives the full coarse-graining and hierarchical formulation.

\section{Method}

\subsection{Teacher search}

Teacher search balances information and cost. A useful score is
\[
\mathrm{Score}(\omega)=
\lambda_{\mathrm{TS}}I_T(\omega)
-
\alpha_{\mathrm{KL}}\frac{K-1}{2K\beta^2}\mathcal E_T(\omega)
+
\rho\,\mathrm{Bal}(\bar T_\omega)
+
\eta\,\mathrm{Fit}(\omega)
-
\xi\,\mathrm{Instab}(\omega).
\]
The first term rewards nontrivial teacher information. The second penalizes log-odds energy. The remaining terms encourage balanced component use, good GMM fit, and stability under seeds or mild perturbations.

\subsection{Training objective}

The main training objective is
\[
\mathcal L_{\mathrm{total}}
=
\mathcal L_{\mathrm{rec}}
+\alpha_{\mathrm{KL}}\mathcal L_{\mathrm{KL}}
+\lambda_{\mathrm{TS}}L_{\mathrm{TS}}.
\]
The teacher and witness enter the optimization only through $L_{\mathrm{TS}}$; the optimized variables are the VAE parameters.

A minimal protocol is:
\begin{enumerate}[leftmargin=1.5em]
    \item search a fixed GMM teacher and compute $T_x$, $\bar T$, $I_T$, $r_k(x)$, and $\mathcal E_T$;
    \item construct the fixed simplex witness $S$;
    \item train the VAE with the weighted alignment term;
    \item report $G_T=I_T-L_{\mathrm{TS}}$ after training.
\end{enumerate}

The optional analytic target $z_T(x)$ can be used for prefit, but the main theorem only needs the final margin. We report three RST variants. \emph{RST} trains from random initialization with the VAE objective plus $\lambda_{\mathrm{TS}}L_{\mathrm{TS}}$. \emph{RST-prefit} first pretrains the encoder mean toward the full analytic target $z_T(x)$ and then runs the same RST objective. \emph{RST-alpha-prefit} instead pretrains toward the minimum scaled target $\alpha^\star z_T(x)$ identified by the feasibility calculation and then runs the same RST objective. The first variant tests the direct optimization effect of the witness loss, the second is a stronger engineering initialization, and the third is the variant most directly tied to the scaled-target feasibility theorem.

\paragraph{Gradient-dominance diagnostic.}
To test whether the function-space escape force remains effective in practical optimization, we also measure
\[
R_{\mathrm{grad}}=
\frac{\lambda_{\mathrm{TS}}\|g_{\mathrm{TS}}\|}{\|g_{\mathrm{VAE}}\|+10^{-12}},
\qquad
\cos(g_{\mathrm{TS}},g_{\mathrm{VAE}}),
\]
near collapsed or low-margin states. A large ratio and a non-opposing cosine indicate that the alignment term is not being neutralized by the VAE gradients. These diagnostics do not replace the final certificate $G_T>0$; they test whether the escape mechanism is visible in the actual optimizer.

\section{Experiments}
\label{sec:experiments}

The experiments test the three claims in the paper: feasibility, training-time certification, and escape. The core metric is always
\[
G_T=I_T-L_{\mathrm{TS}}.
\]
A positive value certifies that the encoder mean is not an exact input-independent constant in the teacher-witness view. Active units, effective rank, reconstruction error, KL, and MMD are reported only as auxiliary diagnostics; they do not constitute the certificate. This distinction matters because a model can have many coordinates with nonzero variance and still fail to beat the teacher constant baseline.

\subsection{Experimental setup}

We use the same convolutional VAE backbone for the natural-image experiments, with latent dimension $d_z=128$. Unless otherwise noted, the main runs use $20$ epochs, $\lambda_{\mathrm{TS}}=1.0$, witness gain $\beta=5.0$, and $5$ prefit epochs for the prefit variants. Each main result is averaged over five seeds. The teacher is fixed before VAE training. The searched CIFAR teachers are unsupervised PCA-GMM teachers; they should be interpreted as nontrivial witness views rather than class-semantic teachers.

\begin{table}[H]
\centering
\caption{Selected teacher statistics. The CIFAR teachers are searched PCA-GMM witness teachers, not class-semantic teachers. The label diagnostics are included only to document that the unsupervised teachers are not simply class labels.}
\label{tab:teacher-stats}
\resizebox{\textwidth}{!}{
\begin{tabular}{llrrrrrrrr}
\toprule
Dataset & Teacher & $K$ & PCA dim & Temp. & $\epsilon$ & $I_T$ & $\mathcal E_T$ & Balance & NMI / ARI \\
\midrule
MNIST & label-smoothed & 10 & -- & -- & -- & 2.0189 & 24.8247 & -- & -- \\
CIFAR-10 & PCA-GMM & 50 & 32 & 0.5 & 0.1 & 2.7275 & 53.5416 & 0.9668 & 0.1229 / 0.0315 \\
CIFAR-100 & PCA-GMM & 50 & 64 & 0.5 & 0.1 & 2.7349 & 57.9823 & 0.9459 & 0.1406 / 0.0166 \\
\bottomrule
\end{tabular}}
\end{table}

\subsection{Controlled MNIST sanity check}

The first experiment uses MNIST with a label-smoothed teacher. This is a controlled sanity check for the certificate, not an unsupervised representation-learning claim. The teacher is
\[
T_{x,k}=(1-\epsilon)\mathbf 1[y=k]+\epsilon/K.
\]
For this run,
\[
I_T=2.0189,
\qquad
\mathcal E_T=24.8247.
\]

\begin{table}[H]
\centering
\caption{MNIST sanity check. The baseline VAE fails the teacher-witness certificate, while the RST-alpha-prefit variant obtains a positive certificate margin.}
\label{tab:mnist-sanity}
\begin{tabular}{lrrr}
\toprule
Method & $I_T$ & $L_{\mathrm{TS}}$ & $G_T$ \\
\midrule
VAE & 2.0189 & 7.0107 & -4.9918 \\
RST-alpha-prefit & 2.0189 & 0.8267 & 1.1922 \\
\bottomrule
\end{tabular}
\end{table}

The MNIST result is a sanity check only. The main evidence below uses searched teachers and five-seed natural-image experiments.

\subsection{Scaled-target feasibility and coarse-view certification}

The scaled-target experiment evaluates the analytic path $\mu_\alpha(x)=\alpha z_T(x)$. We set $\tau=0.1I_T$ and report the smallest $\alpha^\star$ such that
\[
L_{\mathrm{TS}}(\alpha^\star)\le I_T-\tau .
\]
The mean KL cost is the analytic prior cost induced by the scaled target. For each searched $K=50$ teacher we also evaluate a coarse $K=2$ view obtained by merging teacher components. This coarse view should not be read as a class split; it is a group-level witness view. It tests whether a lower-resolution teacher view can already certify nonconstant encoder behavior.

\begin{table}[H]
\centering
\caption{Scaled-target feasibility on searched CIFAR teachers. A very small scaled displacement already beats the exact constant baseline. The coarse $K=2$ view gives a group-level non-collapse certificate at even smaller cost.}
\label{tab:alpha-feasibility}
\begin{tabular}{llrrrrrr}
\toprule
Dataset & View & $K$ & $I_T$ & $\tau$ & $\alpha^\star$ & $G_T(\alpha^\star)$ & Mean KL cost \\
\midrule
CIFAR-10 & full & 50 & 2.7275 & 0.2728 & 0.0570 & 0.2728 & 0.003410 \\
CIFAR-10 & coarse & 2 & 0.3718 & 0.0372 & 0.0402 & 0.0372 & 0.000048 \\
CIFAR-100 & full & 50 & 2.7349 & 0.2735 & 0.0561 & 0.2735 & 0.003576 \\
CIFAR-100 & coarse & 2 & 0.3958 & 0.0396 & 0.0396 & 0.0396 & 0.000051 \\
\bottomrule
\end{tabular}
\end{table}

The full $K=50$ views on CIFAR-10 and CIFAR-100 both require only $\alpha\approx0.056$--$0.057$ to reach the margin threshold. The coarse $K=2$ views require $\alpha\approx0.04$ and almost zero mean KL cost. This supports the feasibility claim: beating the exact constant baseline does not require a large latent displacement.

\subsection{Main five-seed training certificate}

We next train VAE, RST, RST-prefit, and RST-alpha-prefit for five seeds on CIFAR-10 and CIFAR-100. The selected teacher is fixed before all training runs. The main statistic is the final margin $G_T$; we also report the mean final $L_{\mathrm{TS}}$ and the number of seeds with positive margin.

\begin{table}[H]
\centering
\caption{Main five-seed certificate results. Vanilla VAE fails the certificate in all seeds on both datasets. All RST variants obtain positive margins in all seeds.}
\label{tab:main-five-seed}
\resizebox{\textwidth}{!}{
\begin{tabular}{llrrrr}
\toprule
Dataset & Method & Final $G_T$ mean $\pm$ std & Final $L_{\mathrm{TS}}$ mean & Seeds with $G_T>0$ & Interpretation \\
\midrule
CIFAR-10 & VAE & $-1.895 \pm 0.440$ & 4.622 & 0/5 & certificate fails \\
CIFAR-10 & RST & $1.760 \pm 0.010$ & 0.968 & 5/5 & certificate passes \\
CIFAR-10 & RST-prefit & $\mathbf{1.831 \pm 0.032}$ & 0.896 & 5/5 & certificate passes \\
CIFAR-10 & RST-alpha-prefit & $1.700 \pm 0.065$ & 1.027 & 5/5 & certificate passes \\
\midrule
CIFAR-100 & VAE & $-1.720 \pm 0.211$ & 4.455 & 0/5 & certificate fails \\
CIFAR-100 & RST & $1.347 \pm 0.031$ & 1.388 & 5/5 & certificate passes \\
CIFAR-100 & RST-prefit & $\mathbf{1.569 \pm 0.014}$ & 1.166 & 5/5 & certificate passes \\
CIFAR-100 & RST-alpha-prefit & $1.305 \pm 0.044$ & 1.430 & 5/5 & certificate passes \\
\bottomrule
\end{tabular}}
\end{table}

The separation is direct. Vanilla VAE does not pass the teacher-witness test, even though its latent diagnostics are not trivially zero. The RST variants align the latent mean with the fixed witness view and cross the constant-predictor baseline in every seed. RST-prefit gives the largest empirical margins, while RST-alpha-prefit is closest to the constructive scaled-target argument.

\subsection{Collapse-stress experiments}

To test robustness under stronger KL pressure, we run collapse-stress experiments with $\beta_{\mathrm{KL}}\in\{2,4,8\}$. We compare vanilla VAE against RST-alpha-prefit, since the latter is the most direct implementation of the scaled-target feasibility logic. The same searched teachers are used.

\begin{table}[H]
\centering
\caption{Collapse-stress results. Under stronger KL pressure, RST-alpha-prefit margins decrease but remain positive in every seed. Vanilla VAE remains below the certificate threshold in all runs.}
\label{tab:stress}
\resizebox{\textwidth}{!}{
\begin{tabular}{llrrrr}
\toprule
Dataset & Method & $\beta_{\mathrm{KL}}$ & Final $G_T$ mean $\pm$ std & Final $L_{\mathrm{TS}}$ mean & Seeds with $G_T>0$ \\
\midrule
CIFAR-10 & VAE & 2 & $-1.242 \pm 0.154$ & 3.970 & 0/5 \\
CIFAR-10 & RST-alpha-prefit & 2 & $\mathbf{1.473 \pm 0.087}$ & 1.255 & 5/5 \\
CIFAR-10 & VAE & 4 & $-0.764 \pm 0.296$ & 3.492 & 0/5 \\
CIFAR-10 & RST-alpha-prefit & 4 & $\mathbf{1.160 \pm 0.039}$ & 1.568 & 5/5 \\
CIFAR-10 & VAE & 8 & $-0.257 \pm 0.131$ & 2.985 & 0/5 \\
CIFAR-10 & RST-alpha-prefit & 8 & $\mathbf{0.762 \pm 0.046}$ & 1.966 & 5/5 \\
\midrule
CIFAR-100 & VAE & 2 & $-1.219 \pm 0.177$ & 3.954 & 0/5 \\
CIFAR-100 & RST-alpha-prefit & 2 & $\mathbf{1.005 \pm 0.049}$ & 1.730 & 5/5 \\
CIFAR-100 & VAE & 4 & $-0.645 \pm 0.316$ & 3.380 & 0/5 \\
CIFAR-100 & RST-alpha-prefit & 4 & $\mathbf{0.694 \pm 0.024}$ & 2.041 & 5/5 \\
CIFAR-100 & VAE & 8 & $-0.504 \pm 0.318$ & 3.239 & 0/5 \\
CIFAR-100 & RST-alpha-prefit & 8 & $\mathbf{0.318 \pm 0.086}$ & 2.417 & 5/5 \\
\bottomrule
\end{tabular}}
\end{table}

On both natural-image datasets, vanilla VAE fails the certificate in all stress runs. RST-alpha-prefit remains certificate-positive in all runs. The margins shrink as $\beta_{\mathrm{KL}}$ increases, which is expected: the KL term increasingly opposes teacher-directed latent displacement. The important observation is that the RST margin remains strictly positive in every seed.

\subsection{Escape trajectories}

Finally, we test whether the teacher-witness objective provides an optimization direction from a low-margin, near-collapsed initialization. This experiment optimizes the escape objective for 300 steps and records $L_{\mathrm{TS}}$, $G_T$, and the teacher-induced gradient norm. It does not evaluate reconstruction quality; it tests whether the witness force is visible in the local dynamics.

\begin{table}[H]
\centering
\caption{Escape trajectories from low-margin initializations. The margin increases sharply while $L_{\mathrm{TS}}$ decreases, and the teacher-induced gradient norm remains clearly nonzero.}
\label{tab:escape}
\resizebox{\textwidth}{!}{
\begin{tabular}{lrrrrrrrr}
\toprule
Dataset & Steps & $I_T$ & $G_T$ start & $G_T$ end & $G_T$ max & $L_{\mathrm{TS}}$ start & $L_{\mathrm{TS}}$ end & Mean grad. norm \\
\midrule
CIFAR-10 & 300 & 2.7275 & 0.1154 & 2.0003 & 2.1064 & 2.6122 & 0.7272 & 6.1156 \\
CIFAR-100 & 300 & 2.7349 & 0.1122 & 1.6387 & 1.7377 & 2.6228 & 1.0962 & 7.1265 \\
\bottomrule
\end{tabular}}
\end{table}

On CIFAR-10, the margin grows from $0.1154$ to $2.0003$ over 300 steps. On CIFAR-100, it grows from $0.1122$ to $1.6387$. The mean teacher-induced gradient norms are $6.1156$ and $7.1265$, respectively. Thus the teacher-witness force is not numerically degenerate near the low-margin initialization. Because the first recorded margins are already slightly positive, we phrase this result as escape from a low-margin near-collapsed initialization rather than as a negative-to-positive crossing.

\subsection{Summary}

The experiments now match the intended evidence chain:
\begin{enumerate}[leftmargin=1.5em]
    \item \emph{Feasibility:} on CIFAR-10 and CIFAR-100, a small scaled target $\alpha z_T(x)$ crosses the constant baseline with tiny mean KL cost.
    \item \emph{Training certificate:} on both datasets, vanilla VAE fails in all five seeds, while all RST variants pass in all five seeds.
    \item \emph{Stress:} under $\beta_{\mathrm{KL}}\in\{2,4,8\}$, vanilla VAE fails in all stress seeds, while RST-alpha-prefit remains positive.
    \item \emph{Escape:} from low-margin initializations, the witness objective sharply increases $G_T$ and has nonzero gradient norm.
\end{enumerate}
Together these results do not prove optimizer-independent avoidance of every collapse mode. They do show that exact input-independent constant collapse can be made a designable, trainable, and auditable certificate problem.


\section{Limitations and discussion}

The prior remains standard Gaussian, while the aggregate encoder distribution need not be Gaussian. Once the latent mean carries teacher-detectable variation, some mismatch with $\mathcal N(0,I)$ is expected. When unconditional sampling is central, this mismatch should be monitored through teacher smoothing, bounded witness gain, KL budgets, and explicit prior-matching diagnostics.

The escape result is local to the witness alignment component. It rules out exact collapse as a stationary point of that component, but it is not a global convergence theorem for the full nonconvex VAE objective. In practice, reconstruction and KL gradients can oppose the alignment force. The alignment weight $\lambda_{\mathrm{TS}}$ gives a direct control parameter: if an exact cancellation occurs at one value and the teacher-induced gradient is nonzero, changing the weight destroys that cancellation. This motivates reporting $\lambda_{\mathrm{TS}}$ sweeps and gradient-dominance diagnostics, rather than relying on the formal escape statement alone.

The certificate is also a view-level certificate. It concerns the teacher-witness subspace and exact constant collapse of the encoder mean. It does not certify that every latent coordinate, posterior variance, or decoder pathway is used. Reconstruction quality, sampling quality, decoder dependence, per-dimension KL, active units, and linear probes remain separate measurements of representation and generation quality.

Teacher construction has a cost. In this paper the teacher is searched once and then fixed, so the cost is amortized across all RST runs. CPU-based PCA/GMM search is adequate for the moderate-scale experiments considered here. Scaling the same design principle to ImageNet-scale data, stronger feature backbones, or online teacher updates is an engineering and methodological direction beyond the present scope.

More broadly, the work suggests a certificate-and-escape view of anti-collapse representation learning. Such a method specifies a nontrivial view of the input, provides a latent-only witness for that view, exerts a nonvanishing force on the associated collapsed manifold, and leaves a testable margin after training. RST is one realization of this pattern for standard-Gaussian VAEs. Other views, modalities, and world-model latents may admit analogous witness certificates.

\section{Conclusion}

RST combines feasibility, escape, and certification. A fixed simplex witness has a closed-form teacher code, and scaled versions of this code expose the tradeoff between certificate margin and prior proximity. The same alignment loss gives a teacher-residual escape direction at exact collapse. Finally, the margin $L_{\mathrm{TS}}<I_T$ rules out exact input-independent constant collapse of the encoder mean. The experiments instantiate the full chain: searched CIFAR teachers admit small-cost scaled targets, RST variants pass the five-seed certificate on CIFAR-10 and CIFAR-100, RST-alpha-prefit remains positive under KL stress, and escape trajectories show large margin increases from low-margin initializations. The contribution is therefore not an optimizer-independent guarantee for all collapse modes, but a concrete way to design, train, and audit a standard-Gaussian VAE against a specified collapse mode.


\clearpage
\begin{thebibliography}{9}
\bibitem{kingma2014vae}
Diederik P. Kingma and Max Welling.
\newblock Auto-Encoding Variational Bayes.
\newblock ICLR, 2014.

\bibitem{rezende2014stochastic}
Danilo Jimenez Rezende, Shakir Mohamed, and Daan Wierstra.
\newblock Stochastic Backpropagation and Approximate Inference in Deep Generative Models.
\newblock ICML, 2014.

\bibitem{he2019lagging}
Junxian He, Daniel Spokoyny, Graham Neubig, and Taylor Berg-Kirkpatrick.
\newblock Lagging Inference Networks and Posterior Collapse in Variational Autoencoders.
\newblock ICLR, 2019.

\bibitem{razavi2019delta}
Ali Razavi, Aaron van den Oord, and Oriol Vinyals.
\newblock Preventing Posterior Collapse with delta-VAEs.
\newblock ICLR, 2019.

\bibitem{tolstikhin2018wae}
Ilya Tolstikhin, Olivier Bousquet, Sylvain Gelly, and Bernhard Schoelkopf.
\newblock Wasserstein Auto-Encoders.
\newblock ICLR, 2018.

\bibitem{zhang2026teacher}
Zegu Zhang, Jianhua Peng, and Jian Zhang.
\newblock A Testable Certificate for Constant Collapse in Teacher-Guided VAEs.
\newblock arXiv:2605.05813, 2026.
\end{thebibliography}


\clearpage
\appendix

\section{Constructing the simplex vertices}
\label{app:simplex}

The simplex geometry is fixed once $K$ and $d_z$ are chosen. It is not learned. In $\R^K$, let $e_1,\ldots,e_K$ be the standard basis and define
\[
u_k=e_k-\frac1K\mathbf 1.
\]
Then $\sum_k u_k=0$ and the points lie in the $(K-1)$-dimensional hyperplane orthogonal to $\mathbf 1$. After normalization,
\[
\tilde v_k=\frac{u_k}{\|u_k\|},
\]
one obtains
\[
\tilde v_i^\top\tilde v_j=
\begin{cases}
1,&i=j,\\
-\frac1{K-1},&i\ne j.
\end{cases}
\]
If $d_z>K-1$, the simplex is embedded into any fixed $(K-1)$-dimensional subspace of $\R^{d_z}$, with the remaining coordinates set to zero. Thus the difficult design choice is the teacher posterior $T_x$, not the simplex vertices.

\section{Scaled analytic targets and the margin-energy tradeoff}
\label{app:alpha-target}

The closed-form code $z_T(x)$ gives exact matching, $S(z_T(x))=T_x$. It is often preferable to use a scaled target
\[
\mu(x)=\alpha z_T(x),\qquad 0\le \alpha\le1,
\]
which trades certificate strength against proximity to the Gaussian prior center. Along this ray,
\[
S_k(\alpha z_T(x))=
\frac{T_{x,k}^{\alpha}\bar T_k^{1-\alpha}}
{\sum_j T_{x,j}^{\alpha}\bar T_j^{1-\alpha}}.
\]
Thus $\alpha=0$ gives the constant predictor $\bar T$, while $\alpha=1$ gives the teacher $T_x$. Let
\[
Z_\alpha(x)=\sum_j T_{x,j}^{\alpha}\bar T_j^{1-\alpha}.
\]
Then
\[
\KL(T_x\|S(\alpha z_T(x)))
=(1-\alpha)\KL(T_x\|\bar T)+\log Z_\alpha(x).
\]
Hence $L_{\mathrm{TS}}(0)=I_T$ and $L_{\mathrm{TS}}(1)=0$.

The path is continuous. It is also nonincreasing in $\alpha$ on $[0,1]$. To see this, write
\[
\ell_k(x)=\log\frac{T_{x,k}}{\bar T_k},
\qquad
A_x(\alpha)=\log\sum_j \bar T_j\exp(\alpha \ell_j(x)).
\]
Then
\[
S_\alpha(k\mid x)=\bar T_k\exp\{\alpha\ell_k(x)-A_x(\alpha)\},
\]
and the log-partition function satisfies
\[
A_x'(\alpha)=\E_{S_\alpha(x)}\ell(x),\qquad
A_x''(\alpha)=\operatorname{Var}_{S_\alpha(x)}(\ell(x))\ge0.
\]
Thus $A_x'$ is nondecreasing. Since $S_1(x)=T_x$, we have $A_x'(1)=\E_{T_x}\ell(x)$, and therefore $A_x'(\alpha)\le A_x'(1)$ for $0\le\alpha\le1$. Differentiating the KL expression gives
\[
\frac{d}{d\alpha}\KL(T_x\|S(\alpha z_T(x)))
=
-\E_{T_x}\ell(x)+A_x'(\alpha)
\le0.
\]
In nondegenerate cases the inequality is strict except on flat intervals where the relevant log-odds are constant.

For any margin $0<\tau<I_T$, at least one $\alpha$ satisfies
\[
L_{\mathrm{TS}}(\alpha)\le I_T-\tau.
\]
The smallest such value,
\[
\alpha^\star=\inf\{\alpha\in[0,1]:L_{\mathrm{TS}}(\alpha)\le I_T-\tau\},
\]
gives the minimum-energy point on this analytic ray that still achieves the desired certificate margin. Its mean-KL energy is
\[
\frac12\E_x\|\alpha^\star z_T(x)\|^2
=(\alpha^\star)^2\frac{K-1}{2K\beta^2}\mathcal E_T.
\]
The design interpretation is direct: move only as far from the prior center as needed to beat the constant baseline.

The same interpretation can be used to quantify partial non-collapse inside the protected witness subspace. The scalar $\alpha$ does not create new active latent directions; it only scales the teacher directions already present in $z_T(x)$. Thus, for $\alpha>0$, the rank of the mean-code covariance along this ray is the same as the rank of $\operatorname{Cov}(z_T(x))$. What improves continuously is not full dimensional usage, but the amount of teacher information expressed by the fixed witness. This distinction is important: $L_{\mathrm{TS}}\approx0$ means $S(\mu(x))\approx T_x$, not that every coordinate or every posterior dimension is noncollapsed.

\section{Group and hierarchical teacher views}
\label{app:group-hierarchy}

The main theorem applies to a single $K$-component teacher. The same constant-baseline argument can be applied to coarser or hierarchical views derived from the same teacher.

Let $A\in\{0,1\}^{K_A\times K}$ be a fixed grouping matrix whose rows merge disjoint subsets of the $K$ components. Define
\[
T_x^A=A T_x,\qquad \bar T^A=A\bar T.
\]
The information content of this group view is
\[
I_A=\E_x\KL(T_x^A\|\bar T^A).
\]
By the data-processing inequality for KL divergence,
\[
0\le I_A\le I_T.
\]
If $I_A=0$, the group view is constant and gives no group-level certificate. If $I_A>0$, the same simplex construction with $K_A$ vertices gives a group-level witness $S^A$ and a group-level alignment loss
\[
L_A=\E_x\KL(T_x^A\|S^A(z_A(x))).
\]
Then
\[
L_A<I_A
\]
rules out input-independent constant collapse of the group-level code $z_A(x)$.

There are two ways to use this observation. First, one may define a separate latent subspace $z_A$ and a separate group witness. At the collapsed point $z_A(x)\equiv0$, the group-level escape force is
\[
F_A(x)=\beta_A\sum_{b=1}^{K_A}\bigl(T_{x,b}^A-\bar T_b^A\bigr)v_b^A,
\]
which is not identically zero whenever $I_A>0$. This is the direct group analogue of Proposition~1.

Second, if the group prediction is obtained by merging the original student probabilities, $S^A=A S$, then KL contraction gives
\[
L_A=\E_x\KL(A T_x\|A S(\mu_\phi(x)))\le
\E_x\KL(T_x\|S(\mu_\phi(x)))=L_{\mathrm{TS}}.
\]
Thus, a sufficiently strong original margin automatically certifies group views:
\[
L_{\mathrm{TS}}<I_A
\quad\Longrightarrow\quad
L_A<I_A.
\]
Along the analytic $\alpha$-path, each group view therefore has a threshold
\[
\alpha_A^\star
=
\inf\{\alpha\in[0,1]: L_{\mathrm{TS}}(\alpha)< I_A\}.
\]
As $\alpha$ increases, progressively more group views may cross their own constant baselines. This gives a quantitative but limited sense in which partial collapse is improved: the protected teacher information is expressed at more coarse or hierarchical levels. It does not imply that unprotected latent coordinates outside the witness subspace are used.

A hierarchical teacher is obtained by representing the component index as a sequence of decisions,
\[
C=(B_1,B_2,\ldots,B_m).
\]
For a binary hierarchy, $m$ can be of order $\lceil\log_2 K\rceil$. The information decomposes by the chain rule,
\[
I(X;C)
=
I(X;B_1)+I(X;B_2\mid B_1)+\cdots+I(X;B_m\mid B_{<m}).
\]
This gives a structured interpretation of non-collapse: early levels certify coarse teacher information, while later conditional levels certify finer distinctions inside previously selected groups. In this view, partial non-collapse is not a coordinate-wise statement; it is a statement about which teacher views, from coarse to fine, are carried by the latent representation.


\section{Distributions of $z$, $q(z)$, and $z_T(x)$}
\label{app:distributions}

The VAE encoder specifies a conditional Gaussian posterior,
\[
q_\phi(z\mid x)=\N(\mu_\phi(x),\operatorname{diag}\sigma_\phi^2(x)).
\]
Thus $z\mid x$ is Gaussian. The aggregate posterior,
\[
q_\phi(z)=\int q_\phi(z\mid x)p_{\rm data}(x)\,dx,
\]
is generally a mixture of conditional Gaussians and need not equal a single standard Gaussian. This is compatible with the standard VAE setup: the prior is $p(z)=\N(0,I)$, and the ELBO penalizes each conditional posterior through $\KL(q_\phi(z\mid x)\|p(z))$.

The analytic target $z_T(x)$ is the pushforward of the data distribution through the centered log-odds map. It is generally not Gaussian. For label-smoothed teachers it may take finitely many values; for soft GMM teachers it varies continuously with $x$. The certificate is evaluated through the deterministic path $\mu_\phi(x)\mapsto S(\mu_\phi(x))$, because this path is stable and directly tests input dependence of the encoder mean. Sampling quality and aggregate prior matching should be reported separately when unconditional generation is discussed.

\section{Distance to the analytic target and its effect on $L_{\mathrm{TS}}$}
\label{app:target-distance}

Let
\[
d(x)=\mu_\phi(x)-z_T(x).
\]
Since $S(z_T(x))=T_x$, the witness-logit perturbation caused by $d(x)$ is
\[
\delta a_k(x)=\beta v_k^\top d(x).
\]
Therefore
\[
\KL(T_x\|S(z_T(x)+d(x)))
=
\log\sum_jT_{x,j}e^{\delta a_j(x)}-
\sum_kT_{x,k}\delta a_k(x).
\]
For small $d(x)$,
\[
\KL(T_x\|S(z_T(x)+d(x)))
\approx
\frac12\delta a(x)^\top
\left[\operatorname{Diag}(T_x)-T_xT_x^\top\right]
\delta a(x).
\]
A simple upper bound follows from the bounded curvature of the softmax log-partition:
\[
L_{\mathrm{TS}}
\le
\frac{\beta^2K}{2(K-1)}
\E_x\|\mu_\phi(x)-z_T(x)\|^2.
\]
Thus controlling the witness-relevant distance to $z_T(x)$ controls the alignment loss. The converse is not generally true in the full latent space, since directions orthogonal to the simplex span may not change $S$.

\section{Fixed GMM responsibility witnesses}
\label{app:gmm-witness}

The simplex witness can be interpreted as a spherical-GMM responsibility head over the latent space. Let
\[
m_k=a v_k,
\qquad
\Sigma_k=\sigma^2I,
\qquad
\pi_k=\bar T_k.
\]
Then
\[
\frac{\pi_k\N(z;m_k,\Sigma_k)}{\sum_j\pi_j\N(z;m_j,\Sigma_j)}
=\softmax_k\left(\frac{a}{\sigma^2}v_k^\top z+\log\bar T_k\right).
\]
Thus the simplex witness is the most symmetric fixed GMM-responsibility witness, with $\beta=a/\sigma^2$. The latent prior is still $\N(0,I)$; the GMM is used only to define the witness map $S(c\mid z)$.

\section{Component-preserving witness transport}
\label{app:transport}

Component-preserving transport can be used to design or initialize the witness path $T_x\mapsto S(\mu_\phi(x))$. In the standard-Gaussian latent space, this means mapping teacher components to regions where the fixed witness gives the corresponding responsibility. For hard teachers, samples assigned to component $k$ are encouraged to enter the region where $S_k$ is large. For soft teachers, the analytic target $z_T(x)$ gives the exact witness responsibility. This construction is about the teacher-to-witness geometry and does not change the VAE prior.

\section{Gradient dominance and practical variants}
\label{app:gradient-practical}

At exact collapse, the KL gradient with respect to the encoder mean is zero. If the reconstruction gradient transverse to the collapse manifold has norm at most $B$, and the RST escape field has norm at least $\kappa$, then choosing $\lambda_{\mathrm{TS}}\kappa>B$ makes the RST transverse component dominant. In practice, $B$ and $\kappa$ can be estimated with batch gradient norms, and possible cancellation can be monitored with gradient angles.

Analytic prefit can train the encoder mean toward $z_T(x)$ or toward the scaled target $\alpha^\star z_T(x)$ before full VAE training. A margin-constrained variant can also be used in implementation,
\[
\left[L_{\mathrm{TS}}-(I_T-\tau)\right]_+^2.
\]
This implementation device can maintain a chosen margin, while the certificate theorem itself depends only on $G_T=I_T-L_{\mathrm{TS}}$.

\section{Optional reconstruction-side posterior and losses}
\label{app:reconstruction-side}

The main theorem uses only the teacher-student witness path $T_x\to S(\mu_\phi(x))$. If reconstruction-side semantics are of interest, one may define a teacher posterior on the reconstruction,
\[
R_{\hat x}=T_{\omega^\star}(c\mid \hat x),\qquad \hat x=D_\theta(z).
\]
Two optional losses are then natural:
\[
L_{TR}=\E_x\KL(T_x\|R_{\hat x}),
\qquad
L_{SR}=\E_x\KL(S(\mu_\phi(x))\|R_{\hat x}).
\]
The term $L_{TR}$ measures whether the reconstruction preserves the teacher semantics of the input. The term $L_{SR}$ measures whether the latent witness prediction and the reconstruction-side teacher agree. These terms can be useful for diagnostics or quality-oriented extensions, but they do not enter the direct constant-collapse certificate. The certificate remains $G_T=I_T-L_{\mathrm{TS}}>0$.

\end{document}