Unsupervised Brain Anomaly Detection as a Bayesian Inverse Problem with Diffusion Prior

To formalize anomaly detection as a Bayesian inverse problem, we assume that the observed image $\boldsymbol{y}$ is generated from a latent healthy image $\boldsymbol{x}_{0}$ and a latent anomaly mask $\boldsymbol{m}$ through the likelihood $p(\boldsymbol{y}\mid\boldsymbol{x}_{0},\boldsymbol{m})$. Specifically, the observation $\boldsymbol{y}$ is expected to coincide with $\boldsymbol{x}_{0}$ only in regions labeled as normal by the mask $\boldsymbol{m}$. As shown in the graphical model in Fig. 1, assuming prior independence between $\boldsymbol{x}_{0}$ and $\boldsymbol{m}$, the joint distribution factorizes as $p(\boldsymbol{x}_{0},\boldsymbol{m},\boldsymbol{y})=p(\boldsymbol{x}_{0})p(\boldsymbol{m})p(\boldsymbol{y}\mid\boldsymbol{x}_{0},\boldsymbol{m}),$ where $p(\boldsymbol{x}_{0})$ is the prior over healthy images and $p(\boldsymbol{m})$ is a spatial prior over anomaly masks. As in Bayesian inverse problems, the objective is to sample the healthy image $\boldsymbol{x}_{0}$ and anomaly mask $\boldsymbol{m}$ from the posterior $p(\boldsymbol{x}_{0},\boldsymbol{m}\mid\boldsymbol{y})\propto p(\boldsymbol{y}\mid\boldsymbol{x}_{0},\boldsymbol{m})p(\boldsymbol{x}_{0})p(\boldsymbol{m})$. To approximate this posterior, we adopt a similar formulation as in DAPS [25]. Introducing the noisy diffusion states $\boldsymbol{x}_{t_{k}}$ to approximate the image prior $p(\boldsymbol{x}_{0})$, we define a sequence of intermediate posteriors

where $p_{k}(\boldsymbol{x}_{0}\mid\boldsymbol{x}_{t_{k}})$ is the conditional diffusion prior defined in Eq. (1).

Unlike classical inverse problems, the forward model relating a healthy image $\boldsymbol{x}_{0}$ to an abnormal scan $\boldsymbol{y}$ is unknown. In this section, we propose an approximation of the forward model $p(\boldsymbol{y}\mid\boldsymbol{x}_{0},\boldsymbol{m})$ based on the introduced anomaly mask $\boldsymbol{m}$. The probabilistic anomaly mask $\boldsymbol{m}$ encodes pixel-level abnormality: values close to 0 indicate anomalous measurements, whereas values close to 1 enforce consistency between $\boldsymbol{y}$ and $\boldsymbol{x}_{0}$. Accordingly, we model the likelihood as:

with variance $r_{k}^{2}$, matched to the conditional prior $p_{k}(\boldsymbol{x}_{0}\mid\boldsymbol{x}_{t_{k}})$ variance. This ensures that, at intermediate step $k$, in normal regions, the variability of the diffusion-based prediction is consistent with the variability observed in $\boldsymbol{y}$. Deviations exceeding this expected variability cannot be explained by the prior and are therefore attributed to abnormal regions.

Since the probabilistic mask $\boldsymbol{m}$ is introduced as a latent variable, we must specify a prior distribution $p(\boldsymbol{m})$. To enforce values in $(0,1)$, we parametrize it as $\boldsymbol{m}=s(\boldsymbol{a})$, where $\boldsymbol{a}$ are unconstrained logits and $s$ denotes the sigmoid function. We then define a prior $p(\boldsymbol{a})$, that (i) defines the initial mask value, (ii) controls the magnitude of mask updates during inference, and (iii) enforces spatial consistency. To this end, we assume a Gaussian Markov random field prior:

with a precision matrix $\boldsymbol{Q}=\lambda_{0}\boldsymbol{I}+\lambda_{c}\boldsymbol{L}$, where $\lambda_{0},\lambda_{c}>0$ and $\boldsymbol{L}$ is the graph Laplacian of the image grid. The quadratic form $\boldsymbol{a}^{\top}\boldsymbol{L}\boldsymbol{a}$ penalizes differences between neighboring logits, promoting spatial consistency. The parameter $\mu$ sets the initial value of the mask, $\lambda_{0}$ controls the global shrinkage of logits toward $\mu$, and $\lambda_{c}$ governs the spatial consistency strength.

Finally, by combining the conditional diffusion prior defined in Eq. (1), the approximate likelihood in Eq. (3) and the logit mask prior in Eq. (4), the intermediate posteriors in Eq. (2) can be formulated as a Gibbs distribution, i.e. $p_{k}(\boldsymbol{x}_{0},\boldsymbol{a}\mid\boldsymbol{y},\boldsymbol{x}_{t_{k}})\propto\exp(-U_{k}(\boldsymbol{x}_{0},\boldsymbol{a}))$, where $U_{k}(\boldsymbol{x}_{0},\boldsymbol{a})$ is defined as:

In order to sample from the intermediate posteriors, we use unadjusted Langevin dynamics [21] targeting the energy $U_{k}$ at each intermediate step $k$. We denote $j$ the Langevin steps and $(\boldsymbol{x}_{0}^{(j)},\boldsymbol{a}^{(j)})$ the samples at step $j$. At each step $k$, the reconstruction $\boldsymbol{x}_{0}$ is initialized using the mean of the conditional prior sample and the observation. The mask logits are initialized once at $k=0$ as $\boldsymbol{a}^{0}=\mu\boldsymbol{1}$ and are propagated across intermediate steps without re-initialization, enabling progressive refinement of the anomaly mask as the noise level decreases.

where $\boldsymbol{\xi}_{x}^{(j)}$ and $\boldsymbol{\xi}_{a}^{(j)}$ are standard Gaussian noise terms. We use only a few iterations per intermediate step $k$. The step sizes $\eta_{x},\eta_{a}>0$ are automatically adjusted based on gradient magnitudes $\|\nabla_{\boldsymbol{x}_{0}}U_{k}(\boldsymbol{x}_{0}^{(j)},\boldsymbol{a}^{(j)})\|^{2}_{2}$ and $\|\nabla_{\boldsymbol{a}}U_{k}(\boldsymbol{x}_{0}^{(j)},\boldsymbol{a}^{(j)})\|^{2}_{2}$.

The posterior energy in Eq. (5) depends explicitly on the variance schedule $r_{k}$. We parametrize $r_{k}=c\,\sigma_{t_{k}}$, where $\sigma_{t_{k}}$ denotes the diffusion noise level and $c>0$ is a calibration constant estimated on a healthy validation set to ensure stable reconstruction in the absence of anomalies. As $r_{k}$ decreases, the relative strength of the likelihood increases compared to the mask prior since it scales as $1/r_{k}^{2}$. If the prior weight $\lambda_{0}$ is kept fixed across intermediate steps, this changing scale alters the balance between the likelihood and the logit prior. To preserve a stable trade-off across intermediate steps, we adapt $\lambda_{0}$. It is chosen such that the gradient magnitudes from the likelihood and mask value are almost equal.

We adopt the diffusion parameterization of Karras et al. [10] and train the diffusion model exclusively on healthy data. The conditional estimate $\hat{\boldsymbol{x}}_{0}(\boldsymbol{x}_{t_{k}},t_{k})$ in Eq. (1) is obtained using the probability flow ordinary differential equation [20] (PF-ODE) with two solver steps per noise level. For all experiments, we use 150 intermediate posterior steps in the DAPS framework and 75 Langevin iterations at each intermediate step. The noise levels $\{t_{k}\}$ follow the DAPS parameterization with $\sigma_{\max}=20$. We set the spatial consistency parameters $\lambda_{c}$ to 1. An 80GB NVIDIA H100 GPU was used for training.

This work was supported by ANR through the “Investissements d’avenir” program (ANR-10-IAIHU-06, ANR-19-P3IA-0001, PRAIRIE 3IA Institute), the “France 2030” program (ANR-23-IACL-0008, PRAIRIE-PSAI), and the MediTwin project. It also received support from the European Union’s Horizon Europe Framework Programme under grant agreement No. 101136607 (CLARA), and from Next Generation EU through the “France Relance” plan. The ARAMIS Lab is affiliated with DIM C-BRAINS, funded by the Conseil Régional d’Ile-de-France. Experiments used GENCI–IDRIS HPC resources under grants 2025-AD011011648R5 and AD011017165. R.D. received Marie Skłodowska-Curie grant No. 101154248 (SafeREG). Data collection and sharing were funded by the Alzheimer’s Disease Neuroimaging Initiative (ADNI), supported by NIH grant U19AG024904, with additional public and private-sector support.

The authors have no competing interests to declare that are relevant to the content of this article.