FeatMark: Feature-level Watermark Protection against Mimicry Attacks with Diffusion Models

Haoyang Li Affiliation: The Hong Kong Polytechnic University Ruoxi Sun Affiliation: CSIRO’s Data61 Qingqing Ye Affiliation: The Hong Kong Polytechnic University Benjamin Zi Hao Zhao Affiliation: Macquarie University^*Corresponding author: haibo.hu@polyu.edu.hk Yaxin Xiao Affiliation: The Hong Kong Polytechnic University Jason Xue Affiliation: CSIRO’s Data61 Haibo Hu Affiliation: The Hong Kong Polytechnic University

Abstract

Text-to-image diffusion models enable data-efficient “mimicry” attacks, wherein adversaries fine-tune the model on a handful of public photos to synthesize convincing forgeries of a target individual. A common countermeasure is to embed imperceptible, low-energy watermarks, yet recent studies show these signatures are brittle: modest post-processing or lightweight adversarial perturbations readily suppress detection, exposing a fundamental tension between imperceptibility and robustness. We introduce FeatMark, a watermarking framework that shifts from pixel-level, energy-starved perturbations to inconspicuous semantic features: small, scene-consistent micro-features that remain natural to humans while providing a stronger, machine-verifiable provenance signal. FeatMark builds domain-specific feature banks that encode each watermark as a compact concept program, pairing open-vocabulary semantic cues with reliable edit regions and instruction templates. It then automatically selects features that are both feasible and executable and injects them through modular, mask-guided concept editing, yielding highly localized, scene-consistent micro-edits that are difficult to perceive. We conduct extensive experiments across VGGFace2, CelebA-HQ, and WikiArt, evaluating against 10 strong watermark removal/purification attacks (including regeneration-style purification) and several bespoke adaptive attacks tailored to FeatMark, to assess perceptual fidelity, watermark detection accuracy, and robustness. We further demonstrate FeatMark’s extensibility to video mimicry attacks. The results show FeatMark remains virtually impervious, withstanding all evaluated attacks with negligible bit-accuracy and fidelity degradation.

1 Introduction

Text-to-image diffusion models such as Stable Diffusion [67] and DALL-E [63] empower creative synthesis but also make mimicry attacks feasible [64, 36, 14], in which an adversary fine-tunes a pre-trained generator on just a handful of a victim’s public photos so the model learns to replicate that person’s distinctive appearance [68, 56, 83], as Figure 1 shows. Watermarks offer a direct countermeasure, embedding persistent provenance signals into those fake images so that every derivative remains traceable. Recent regulations (e.g. EU AI Act Art. 50 [16] and US EO 14110 [34]) encourage reliable watermarking, underscoring the need for robust provenance. Regarding these regulations, contemporary methods such as Tree-Ring Watermarks [82] and FT-Shield [12] hide imperceptible signatures in attack results by regulating diffusion models or perturbing public images.

Refer to caption — Figure 1: An illustration of mimicry attacks.

However, extant watermarking schemes are demonstrably brittle under adversarial manipulation. Most state-of-the-art watermarking methods [94, 18, 82, 13, 12] insert low-amplitude noise that is invisible to humans yet statistically detectable by a verifier. Although this preserves visual fidelity, it grants attackers a powerful lever: small, human-invisible adversarial perturbations can overwhelm or erase the signature without perceptual cost [29, 33, 28]. Even worse, the adversarial perturbations required for such evasion are computationally and perceptually cheap. For instance, WEvade [33] achieves perfect bypass with imperceptible $\ell_{\infty}$ -bounded noise in only a few PGD iterations. These empirical findings reveal a fundamental trade-off: as watermark energy is reduced to maintain imperceptibility, every incremental gain in stealth tends to reduce the watermark’s energy budget for robustness, perpetuating an arm race that defenders cannot win with the current paradigm [46, 22, 89]. Consequently, it becomes imperative to develop a fundamentally different watermarking mechanism that does not rely solely on imperceptibility and thus cannot be trivially nullified by lightweight adversarial perturbations.

To overcome this impasse, we introduce the concept of inconspicuous-feature watermarks. Instead of hiding energy-starved pixel perturbations, we affix semantically coherent micro-features (e.g. a faint botanical emblem subtly woven into existing foliage) that are perceptible on close inspection yet blend naturally with the generated scene. Their greater signal strength lets them withstand adversarial perturbations while still escaping casual human notice. This principle has clear precedents in security. Patch-based adversarial examples use naturalistic artifacts to mislead detectors [79, 24, 9], and semantic backdoors leverage real objects to control model outputs [2, 74, 80]. These generative-prior features often go unnoticed yet remain effective, motivating our inconspicuous-feature strategy. More importantly, in the scenario of mimicry attack, stealth of inconspicuous feature is aided by an identity-based knowledge gap and diffusion stochasticity [30], which make the features hard to anticipate and easy to ascribe to sampling variance.

Building on these insights, we propose FeatMark, a fully automatic framework that injects inconspicuous semantic micro-features into images and transfers them through mimicry training via image cloaking. FeatMark first constructs domain-specific feature banks, where each entry is a compact concept program that bundles open-vocabulary semantic cues (for recognizing concept evidence), a reliable region-of-edit specification, and instruction templates that a generic editor can follow. Given a target identity or artistic style, FeatMark automatically scores candidate concepts using open-vocabulary recognition and selects a small set of executable features, then performs mask-guided concept-based local editing to realize localized and scene-consistent micro-edits. Crucially, FeatMark does not publish the edited images directly. Instead, it produces the released images by cloaking the original images toward their edited counterparts, so that the published set remains visually faithful while behaving like the watermarked set under mimicry training representations. Finally, a private watermark reader is trained to decode a user-specific message from suspicious images, enabling attribution of mimicry outputs without requiring access to the attacker’s model.

We evaluate FeatMark across both facial and artistic domains, including VGGFace2 [8], CelebA-HQ [35], and WikiArt [70]. Across datasets, FeatMark preserves perceptual quality close to the no-defense baseline while providing a verifiable watermark signal with high decoding accuracy on watermarked images and near-random decoding on clean images. We further test robustness against a broad suite of strong purification and watermark removal attacks, including IMPRESS [6], Noisy Upscaling [26], WEvade [33], UnMarker [37], regeneration-style removals (diffusion/denoise [59] and VAE [3]), and WatermarkAttacker [92]. In settings where pixel-level invisible watermarks collapse, FeatMark’s decoding remains essentially unchanged under most attacks (typically a $\leq 0.02$ drop), and even under stronger regeneration-style purification the accuracy degradation largely coincides with visible fidelity loss, leaving FeatMark still decodable. To probe future, attacker-tailored strategies, we additionally design bespoke adaptive attacks targeted to FeatMark’s pipeline and show that removing the watermark without noticeable identity/style drift remains difficult. Finally, we demonstrate that the same feature-anchored design transfers to video mimicry.

To summarize, our key contributions are as follows:

•

To our knowledge, FeatMark is the first watermarking scheme that protects against mimicry attacks by anchoring provenance in inconspicuous semantic micro-features rather than energy-starved pixel perturbations.
•

We introduce a fully automatic watermarking pipeline built on feature-bank concept programs, automatic executable feature selection, and mask-guided concept-based local editing, coupled with a cloaking stage that enables safe publishing while preserving watermark transfer through mimicry training.
•

We provide comprehensive evaluations over perceptual fidelity, watermark decoding accuracy, and robustness to strong purification, removal, and regeneration attacks, and further assess attacker-tailored adaptive attacks designed for FeatMark.

2 Background

Mimicry attacks. We study mimicry attacks where an adversary exploits a small set of publicly released personal images of a target individual, denoted as $\mathcal{X}=\{x\}$ , to fine-tune a generative model and synthesize high-fidelity impersonating images. Following prior work on unauthorized personalization and style/identity mimicry [6, 72, 68], we model the attacker as fine-tuning a text-to-image diffusion model $\mathcal{M}^{\theta}$ and sampling fake images $x_{\text{fake}}$ conditioned on prompt $c$ . The attacker optimizes:

\theta^{*}=\arg\max_{\theta}\ \mathbb{E}_{\begin{subarray}{c}x_{\text{fake}}\sim\,p(x_{\text{fake}}\mid c,\mathcal{M}^{\theta})\end{subarray}}\!\big[\mathcal{A}(x_{\text{fake}},\mathcal{X})\big],

(1)

where $\mathcal{A}$ is an attacker-chosen similarity/utility metric that encourages the synthesized outputs to match the target identity or style.

Attribution via watermarking. Our goal is attribution rather than disruption: the defender transforms their released images such that mimicry outputs tend to carry a user-specific signature that can later be verified. Concretely, the defender holds a private message $m$ and a watermark reader $\mathcal{R}$ that decodes a bit sequence from a suspicious image. We aim for high expected bit agreement between $m$ and $\mathcal{R}(\cdot)$ on attacker-generated samples:

	$\displaystyle\max$	$\displaystyle\mathbb{E}_{x_{\text{fake}}\sim p(x_{\text{fake}}\mid c,\mathcal{M}^{\theta^{*}})}\big[\mathcal{B}_{\text{acc}}(m,\mathcal{R}(x_{\text{fake}}))\big]$		(2)
	$\displaystyle\text{s.t. }$	$\displaystyle\theta^{*}\text{ satisfies Eq.~\ref{eq:attack_obj}}.$		(2)

Here, $\mathcal{B}_{\text{acc}}$ measures the fraction of correctly recovered bits.

Adaptive attackers. A defender-facing evaluation must consider adaptive attackers who post-process outputs to evade verification while preserving identity/style. We model such post-processing by an attacker-side transformation $\mathcal{P}$ , yielding the adaptive objective:

\displaystyle\theta^{*}=\arg\max_{\theta}

\displaystyle\mathbb{E}_{\begin{subarray}{c}x_{\text{fake}}\sim\,p(x_{\text{fake}}\mid c,\mathcal{M}^{\theta})\end{subarray}}\!\big[\mathcal{A}(\mathcal{P}(x_{\text{fake}}),\mathcal{X})\big].

(3)

This motivates designing watermarking mechanisms that are robust to unknown, potentially strong adaptive strategies.

3 Methodology

This section specifies the threat model, summarizes the end-to-end workflow, and details each module of FeatMark.

3.1 Threat Model

We follow threat models in prior work on protecting personal data from malicious personalization [72, 71, 77, 44]. The defender is an individual user who publishes a small set of images and later wishes to attribute whether a suspicious image was generated by a mimicry model trained on those released images. The defender holds a private target message $m$ and runs a private watermark reader $\mathcal{R}$ locally, which are not exposed to the attacker.

The attacker has access to the published images and sufficient compute to fine-tune a text-to-image diffusion model $\mathcal{M}^{\theta}$ to generate impersonating outputs, as in Eq. 1. We assume the attacker knows the defense algorithm (including the existence of feature-bank-driven watermarking), and may deploy advanced strategies such as purification $\mathcal{P}$ or regeneration-based transformations to evade verification, but does not have access to the defender’s private verification parameters.

3.2 Overview of Our Watermark Workflow

FeatMark is a fully automatic pipeline that converts a small personal image set into publishable training data that transfers a user-specific semantic watermark through mimicry training. Starting from the original set $\mathcal{X}=\{x\}$ , FeatMark queries a Feature Bank $\mathcal{F}=\{f_{k}\}_{k=1}^{K}$ whose entries are executable concept programs. Each concept program bundles textual descriptors for open-vocabulary concept evidence, grounding prompts for localizing an editable region, and instruction templates for expressing the concept via a generic editor. This design removes the need for manual feature crafting and manual masking.

Given $\mathcal{X}$ , FeatMark automatically selects a feature that is semantically suitable for the set and operationally executable. It then performs localized, instruction-guided concept editing to obtain a privately held watermarked set $\mathcal{X}^{\mathrm{wm}}=\{x^{\mathrm{wm}}\}$ , where each $x^{\mathrm{wm}}$ differs subtly from its corresponding $x$ while reliably expressing the selected concept.

Crucially, the images that are actually published are produced by an image cloaking stage. For each original image $x$ , FeatMark computes a released image $x^{\mathrm{pub}}$ by adding an imperceptible perturbation so that $x^{\mathrm{pub}}$ remains visually close to $x$ but is optimized to behave like $x^{\mathrm{wm}}$ under the representations used by mimicry training, yielding a released set $\mathcal{X}^{\mathrm{pub}}=\{x^{\mathrm{pub}}\}$ . Only $\mathcal{X}^{\mathrm{pub}}$ is shared with the public and thus accessible to an attacker for fine-tuning, while $\mathcal{X}^{\mathrm{wm}}$ is kept private and later used as supervision. This cloaking step follows the perturbation-based protection paradigm in prior work [72, 71] but is integrated here to facilitate semantic watermark transfer rather than merely disrupting generation.

After generating the publishable set, the defender trains a private watermark reader $\mathcal{R}$ using a small number of clean images from $\mathcal{X}$ and their corresponding privately stored watermarked counterparts from $\mathcal{X}^{\mathrm{wm}}$ . At test time, given a suspicious image, the defender decodes a bit sequence with $\mathcal{R}$ and verifies agreement with the user’s private message $m$ , enabling attribution of mimicry outputs generated from models fine-tuned on $\mathcal{X}^{\mathrm{pub}}$ .

3.3 Feature Bank Construction

FeatMark builds on a feature bank, a library of watermarkable concepts that can be consistently recognized from an image and later reproduced by a mimic generator. Each entry $f_{k}\in\mathcal{B}$ is specified as a compact concept program that defines how the concept is queried and described. Concretely, $f_{k}$ provides a small set of textual queries $\mathcal{T}_{k}=\{t_{k,1},\dots,t_{k,J}\}$ (paraphrases/templates) that all refer to the same concept, and a canonical name $\text{name}(f_{k})$ that will later be used to form an editing instruction. The guiding intuition is to treat a feature as an atomic semantic attribute whose presence leaves a stable trace in representation space, while remaining editable with minimal perceptual disruption.

To measure concept evidence, we use an open-vocabulary vision-language model [62] to map an image $x$ and a text query $t$ into a joint embedding space. We define the evidence score of feature $f_{k}$ on image $x$ as

s_{k}(x)\;=\;\max_{t\in\mathcal{T}_{k}}\ \cos\!\big(\phi_{I}(x),\,\phi_{T}(t)\big),

(4)

where $\phi_{I}(\cdot)$ and $\phi_{T}(\cdot)$ denote the image and text encoders, and $\cos(\cdot,\cdot)$ is cosine similarity. Taking the maximum over paraphrases follows the standard prompt-ensembling practice for reducing prompt sensitivity in open-vocabulary recognition [62, 93]. Importantly, Eq. (4) only requires a generic VLM interface and does not rely on a specific backbone.

Our bank construction follows two principles with theoretical support from feature selection and representation learning. First, each feature’s evidence should be stable across natural variations of the same identity or style, which encourages downstream generators to internalize and reuse the cue. This is aligned with the well-documented tendency of modern models to exploit stable, high-signal shortcuts when such cues remain predictive in-distribution [21]. Second, the bank should be diverse and non-redundant, since near-duplicate concepts waste watermark capacity and make automatic selection brittle. We therefore curate $\mathcal{B}$ to cover multiple semantic axes and later enforce redundancy control during selection using classical relevance, redundancy criteria from information-theoretic feature selection [23, 60, 5]. These principles keep the bank compact, interpretable, and amenable to fully automatic downstream selection.

3.4 Automatic Feature Selection from the Bank

Given a target set of images $\mathcal{X}=\{x_{i}\}_{i=1}^{n}$ for one identity or style, FeatMark selects a small subset of features $\mathcal{K}\subset\{1,\dots,|\mathcal{B}|\}$ without any manual screening. The selection aims to identify concepts that are consistently detectable on $\mathcal{X}$ and also complementary to each other, so that watermark bits are distributed over multiple independent semantic traces rather than concentrated on a single fragile cue.

We first compute evidence statistics for each bank feature using Eq. (4). For feature $f_{k}$ , we aggregate scores across the set,

	$\displaystyle\mu_{k}$	$\displaystyle=\frac{1}{n}\sum_{i=1}^{n}s_{k}(x_{i}),$		(5)
	$\displaystyle\sigma_{k}^{2}$	$\displaystyle=\frac{1}{n}\sum_{i=1}^{n}\big(s_{k}(x_{i})-\mu_{k}\big)^{2},$		(5)

and keep candidates with sufficiently high mean evidence and low variance. This filter-style step favors stable cues over spurious ones and follows standard practice in feature selection [23]. To avoid selecting multiple semantically overlapping features, we further apply redundancy-aware pruning. Let $\psi_{k}$ denote a textual embedding of $\text{name}(f_{k})$ (or the centroid of $\{\phi_{T}(t):t\in\mathcal{T}_{k}\}$ ). We penalize candidates whose textual representations are too similar to already selected ones and greedily grow $\mathcal{K}$ by maximizing a relevance–redundancy trade-off, an unsupervised analogue of mRMR-style criteria [60, 5].

Because this procedure depends only on the bank and $\mathcal{X}$ , it enables a fully automatic pipeline while still producing a small set of semantically distinct, high-confidence concepts. In practice, the resulting $\mathcal{K}$ reduces downstream editing failures and improves robustness by preventing the watermark from collapsing into a single easily targeted attribute.

3.5 Concept-Based Image Editing

Once a feature $f_{k}$ is selected, FeatMark performs concept-based editing to inject or modify the corresponding semantic cue while keeping the output realistic and identity/style-consistent. The concept program provides a grounding phrase $\text{ground}(f_{k})$ that specifies where the concept should be edited (e.g., an attribute-associated region), and an instruction template $\text{inst}(f_{k},\cdot)$ that specifies how it should be edited (e.g., add or strengthen the concept).

We obtain a region-of-edit mask by applying an open-world grounding module to localize $\text{ground}(f_{k})$ on image $x$ , producing a soft mask $M_{k}(x)\in[0,1]^{H\times W}$ . The instruction is instantiated as $u_{k}=\text{inst}(f_{k},\text{name}(f_{k}))$ . A generic concept-guided editor $\mathcal{E}$ then produces an edited image

x_{k}^{\text{edit}}\;=\;\mathcal{E}\!\big(x,\,u_{k},\,M_{k}(x)\big).

(6)

Our methodology only requires $\mathcal{E}$ to support localized, instruction-following edits, and it can be instantiated by a wide range of modern image editing frameworks, including diffusion-based editors [54, 25, 11, 4].

To discourage unnecessary changes outside the edited region, we use a mask-weighted consistency objective during editing (or as a post-selection validation),

\displaystyle\mathcal{L}_{\text{out}}(x,x_{k}^{\text{edit}})

\displaystyle=\big\|(1-M_{k}(x))\odot(x_{k}^{\text{edit}}-x)\big\|_{1}.

(7)

This formulation makes the watermark signal semantic: bits correspond to the presence or strength of concepts that are edited in a localized and visually plausible manner, which later yields robustness against perturbation-level removals.

3.6 Image Cloak with Refined Objective

In FeatMark, image cloaking is a required stage for producing publishable images. After concept-based editing yields a watermarked set $\mathcal{X}^{\mathrm{wm}}=\{x^{\mathrm{wm}}\}$ , we further transform each original image $x$ into a released image $x^{\mathrm{pub}}$ that remains visually faithful to $x$ while being optimized to transfer the watermark feature through mimicry training. Concretely, only $\mathcal{X}^{\mathrm{pub}}=\{x^{\mathrm{pub}}\}$ is published and thus accessible to the attacker for fine-tuning, whereas $\mathcal{X}^{\mathrm{wm}}$ is retained privately and will be used in the subsequent watermark reader training as supervision.

Following perturbation-based cloaking methods that add imperceptible noise to influence downstream generative behavior [72, 71], we parameterize the released image as $x^{\mathrm{pub}}=x+\delta$ and optimize $\delta$ so that $x^{\mathrm{pub}}$ stays close to $x$ in pixel space yet aligns with the edited target $x^{\mathrm{wm}}$ in a representation space. Specifically, we refine the standard cloaking objective with a reconstruction-consistency regularizer to reduce overfitting to a single encoder and to avoid degenerate perturbations:

$\displaystyle\mathcal{L}_{\text{cloak}}={}$	$\displaystyle\left\\|\mathfrak{E}(x+\delta)-\mathfrak{E}(x^{\text{wm}})\right\\|_{2}^{2}$	(8)
	$\displaystyle+\lambda\left\\|(x+\delta)-\mathfrak{D}(\mathfrak{E}(x+\delta))\right\\|_{2}^{2},$
$\displaystyle\text{s.t.}$	$\displaystyle\\|\delta\\|_{1}\leq\eta,\quad\forall\delta\in\{\delta\},$

where $\mathfrak{E}(\cdot)$ and $\mathfrak{D}(\cdot)$ denote the encoder and decoder, respectively. The first term encourages the released image to be close to the watermarked target in the encoder space, which promotes watermark feature transfer when the attacker fine-tunes on $\mathcal{X}^{\mathrm{pub}}$ . The second term regularizes the perturbation to remain stable under an encode–decode cycle, mitigating brittle solutions that can arise from purely feature-space alignment [26, 6].

Table 1: Results on perceptual quality for all datasets. For each metric, we calculate the results between fake images affected by the corresponding method and original images. Prec., Rec., Cov., and Dens. denote precision, recall, coverage, and density.

Method FID $\downarrow$ Prec. $\uparrow$ Rec. $\uparrow$ Cov. $\uparrow$ Dens. $\uparrow$ VGGFace2 No-Defense 171.81 0.65 0.28 1.33 0.83 FeatMark 180.98 0.47 0.19 0.83 0.62 MetaCloak 457.10 0.00 0.00 0.00 0.00 Glaze 325.89 0.01 0.00 0.01 0.01 Anti-DreamBooth 449.74 0.00 0.00 0.00 0.00 CelebA-HQ No-Defense 128.99 0.54 0.27 1.00 0.68 FeatMark 142.33 0.31 0.18 0.44 0.41 MetaCloak 437.92 0.00 0.00 0.00 0.00 Glaze 279.13 0.01 0.00 0.03 0.01 Anti-DreamBooth 465.37 0.00 0.00 0.00 0.00 WikiArt No-Defense 306.02 0.69 0.13 0.68 0.46 FeatMark 321.76 0.62 0.07 0.36 0.29 MetaCloak 458.61 0.00 0.00 0.00 0.00 Glaze 410.93 0.00 0.00 0.00 0.00 Anti-DreamBooth 447.03 0.00 0.00 0.00 0.00

Table 2: Bit accuracy for watermark methods on all datasets with/without watermarks and applied with multiple transformations. The column of WM? denotes whether the evaluated images contain the watermark or not.

Method WM? Bit Accuracy None Crop Brigh. Cont. JPEG Comb. VGGFace2 Stable Signature Y 0.99 0.36 0.98 0.97 0.66 0.80 N 0.45 0.40 0.49 0.49 0.46 0.46 HiDDeN Y 0.99 0.58 0.97 0.94 0.83 0.62 N 0.43 0.47 0.42 0.45 0.41 0.45 StegaStamp Y 0.98 0.54 0.98 0.97 0.92 0.85 N 0.45 0.46 0.42 0.42 0.42 0.47 FeatMark Y 0.98 0.75 0.96 0.96 0.99 0.98 N 0.03 0.04 0.04 0.04 0.04 0.05 CelebA-HQ Stable Signature Y 0.99 0.38 0.98 0.97 0.66 0.82 N 0.46 0.42 0.49 0.48 0.49 0.46 HiDDeN Y 0.95 0.61 0.90 0.95 0.81 0.67 N 0.45 0.46 0.42 0.42 0.42 0.47 StegaStamp Y 0.95 0.58 0.91 0.98 0.90 0.90 N 0.41 0.50 0.41 0.47 0.38 0.45 FeatMark Y 0.92 0.31 0.90 0.89 0.94 0.88 N 0.16 0.02 0.28 0.22 0.19 0.22 WikiArt Stable Signature Y 0.99 0.50 0.80 0.88 0.72 0.76 N 0.47 0.56 0.46 0.53 0.48 0.52 HiDDeN Y 0.99 0.52 0.96 0.98 0.78 0.62 N 0.41 0.50 0.41 0.47 0.38 0.45 StegaStamp Y 0.99 0.48 0.97 0.99 0.88 0.85 N 0.44 0.45 0.45 0.44 0.41 0.43 FeatMark Y 0.99 0.26 0.77 0.96 0.99 0.91 N 0.01 0.02 0.01 0.01 0.01 0.02

3.7 Watermark Reading

The watermark reader $\mathcal{R}$ determines whether a suspicious image carries the user’s target message $m$ . Unlike pixel-level invisible watermark decoders [94, 18], our watermark signal is semantic and may vary in its localized realization across images; accordingly, we adopt a feature-based reader that leverages a strong frozen image encoder and trains only a lightweight head.

Concretely, we instantiate $\mathcal{R}$ as a frozen CLIP image encoder [62] followed by a trainable fully connected layer that outputs a $|m|$ -bit vector. Given a small set of clean images $\mathcal{X}=\{x\}$ , their watermarked counterparts $\mathcal{X}^{\text{wm}}=\{x^{\text{wm}}\}$ , and the target message $m$ , we train $\mathcal{R}$ with:

	$\displaystyle\mathcal{L}_{\text{read}}={}$	$\displaystyle\mathbb{E}_{x\sim\mathcal{X}}\big[\mathrm{BCE}(\bar{m},\mathcal{R}(x))\big]$		(9)
		$\displaystyle+\mathbb{E}_{x^{\text{wm}}\sim\mathcal{X}^{\text{wm}}}\big[\mathrm{BCE}(m,\mathcal{R}(x^{\text{wm}}))\big],$		(9)

where $\bar{m}$ denotes the bitwise complement of $m$ . During both training and inference, standard augmentations (cropping, rotation, mild noise) and weight decay are applied to improve generalization. After training, verification reduces to checking whether the decoded bits from a suspicious image match $m$ above a fixed threshold, enabling attribution of attacker-generated outputs even when the attacker applies post-processing, provided that the semantic watermark evidence persists through the mimicry pipeline.

4 Evaluations

In this section, we conduct the experiments regarding perceptual quality evaluation, the watermark detection accuracy, and the robustness of our watermarking method against removal and purification attacks. Additional adaptive attacks, ablations, and generalization results appear in Appendices E, H, and G. To ensure clarity, in visualization figures we present four samples per set for each row by default, unless otherwise specified.

4.1 Experimental Setup

We evaluate FeatMark on watermark decoding, perceptual quality, and robustness under purification/removal attacks across VGGFace2, CelebA-HQ, and WikiArt. The implementation uses CLIP-based evidence scoring, text-guided open-world segmentation, and masked diffusion editing; mimicry attacks follow a DreamBooth-style protocol, and metrics include bit accuracy, FID, precision/recall, and coverage/density. We keep the main evaluation focused on the primary results and provide full dataset, implementation, metric, baseline, and user-study details in Appendix A.

Table 3: Bit accuracy change of FeatMark against IMPRESS across all datasets.

Dataset Change of Bit Accuracy None Crop Brigh. Cont. JPEG Comb. VGGFace2 $+$ 0.00 $+$ 0.01 $+$ 0.00 $+$ 0.00 $+$ 0.00 $+$ 0.00 CelebA-HQ $-$ 0.00 $-$ 0.01 $-$ 0.00 $-$ 0.01 $-$ 0.00 $-$ 0.00 WikiArt $-$ 0.00 $+$ 0.00 $+$ 0.00 $-$ 0.01 $-$ 0.00 $-$ 0.00

Table 4: Bit accuracy change of FeatMark against Noisy Upscaling across all datasets.

Dataset Change of Bit Accuracy None Crop Brigh. Cont. JPEG Comb. VGGFace2 $-$ 0.20 $-$ 0.01 $-$ 0.26 $-$ 0.24 $-$ 0.31 $-$ 0.26 CelebA-HQ $-$ 0.22 $-$ 0.02 $-$ 0.24 $-$ 0.19 $-$ 0.26 $-$ 0.30 WikiArt $-$ 0.23 $-$ 0.01 $-$ 0.27 $-$ 0.21 $-$ 0.29 $-$ 0.25

Table 5: Bit accuracy changes of three watermarking schemes in VGGFace2 against three watermark removal attacks, UnMarker, Diffusion Attack, and VAE Attack.

UnMarker Diffusion Attack VAE Attack Stable Signature $-$ 0.99 $-$ 0.99 $-$ 0.99 Tree-Ring $-$ 0.39 $-$ 0.21 $-$ 0.09 FeatMark $-$ 0.01 $-$ 0.01 $-$ 0.00

Table 6: Bit accuracy changes of four watermarking schemes in VGGFace2 against four watermark removal attacks in Watermark Attacker (WMA).

WMA-Diff60 WMA-Cheng3 WMA-BMSHJ3 WMA-JPEG50 DwtDct $-$ 0.33 $-$ 0.33 $-$ 0.30 $-$ 0.29 DwtDctSvd $-$ 0.45 $-$ 0.47 $-$ 0.43 $-$ 0.20 RivaGAN $-$ 0.45 $-$ 0.35 $-$ 0.39 $-$ 0.02 FeatMark $-$ 0.02 $-$ 0.00 $-$ 0.00 $-$ 0.01

4.2 Perceptual Quality Evaluation

Perceptual quality of FeatMark images is measured in terms of the average FID, precision, recall, coverage, and density for all target persons. These are comparative metrics between a source and target data sample, as such for each target identity, the generated fake images impacted by our watermark is contrasted against the original images of that identity. As the name suggests, No-Defense is calculated with fake images that had no defense applied.

Perceptual results. Table 1 shows that FeatMark preserves substantially higher perceptual quality than disruption-based defenses across all datasets. Compared to the No-Defense setting, FeatMark induces only a modest degradation in distributional fidelity: the FID increases by $9.17$ on VGGFace2 ( $180.98$ vs. $171.81$ ), $13.34$ on CelebA-HQ ( $142.33$ vs. $128.99$ ), and $15.74$ on WikiArt ( $321.76$ vs. $306.02$ ). Importantly, FeatMark maintains non-trivial Precision/Recall as well as Coverage/Density, indicating that the generated samples remain both realistic and diverse under our fully automatic semantic watermark injection.

In contrast, existing image cloaking methods severely damage perceptual quality. For examples, FIDs for Anti-Dreambooth and Glaze balloon to $449.74$ – $465.37$ on VGGFace2/CelebA-HQ and $410.93$ on WikiArt, while Precision, Recall, Coverage, and Density collapse to zero. This stark gap highlights FeatMark’s key practical advantage: it enables reliable downstream attribution while keeping the attacker’s outputs close to the original data distribution, rather than relying on heavy distortions that break visual fidelity.

Visualization of FeatMarks. Figures 3 and 4 show VGGFace2 and WikiArt results; CelebA-HQ examples appear in Appendix C. Across VGGFace2, CelebA-HQ, and WikiArt, the visual examples show the same pattern: released cloaked images remain close to the targets, while attackers trained on cloaked images consistently inherit the injected semantic attribute.

User study results. We compare 100 pairs of attacker-generated images from No-Defense and FeatMark using 49 participants. Each pair uses the same target and prompt, with randomized left–right ordering. Participants judge realism, identity/style match, and agreement with the edit instruction. The study protocol is detailed in Appendix A.

Table 7: User study on attacker-generated images under No-Defense vs FeatMark. Each entry is the FeatMark win rate in a two-alternative forced choice test with randomized left-right ordering. For realism and identity or style match, results closer to 50% indicate near-random guessing and therefore higher perceptual indistinguishability. For instruction match, higher win rate indicates stronger and more consistent concept transfer.

Dataset Realism win rate Identity or style match win rate Instruction match win rate VGGFace2 $47.8\%\pm 2.6$ $50.6\%\pm 2.5$ $83.1\%\pm 1.9$ CelebA-HQ $49.2\%\pm 2.5$ $51.3\%\pm 2.4$ $80.4\%\pm 2.1$ WikiArt $48.5\%\pm 2.7$ $49.8\%\pm 2.6$ $77.6\%\pm 2.4$

Table 7 shows that FeatMark remains close to the No-Defense baseline for realism and identity/style match, with win rates near 50%, while strongly improving instruction match. This supports the intended tradeoff: the generated images remain perceptually comparable to the baseline, but the injected concept is reliably absorbed and reproduced during mimicry fine-tuning.

4.3 Watermark Accuracy Evaluation

Table 2 reports bit accuracy under common transformations. FeatMark maintains high decoding reliability on watermarked images and a large verification margin against non-watermarked images: on VGGFace2 it reaches 0.99/0.98 under JPEG/combined transformations while keeping WM=N accuracy at 0.04/0.05, and on WikiArt it reaches 0.99/0.91 while keeping WM=N accuracy at 0.01/0.02. Cropping remains the most challenging case because it can remove the edited region, but FeatMark still preserves substantially cleaner separability than pixel-level baselines; additional discussion is provided in Appendix B.

4.4 Watermark Robustness against Adversarial Purification and Watermark Removal

We stress-test FeatMark against adversarial purification, white-box perturbation attacks, and black-box removal/regeneration attacks. Tables 4–4 summarize the purification results; Tables 6 and 6 report black-box removal results, with the Noisy Upscaling visualization in Appendix C. Attack configurations are described in Appendix A.

Purification attacks. IMPRESS causes essentially no bit-accuracy change across datasets, while Noisy Upscaling produces a measurable but non-decisive reduction of about $0.20$ – $0.31$ under most transformations. Even after Noisy Upscaling, FeatMark remains decodable from its high baseline accuracy, and the reduction largely coincides with visible rewriting artifacts.

Removal and regeneration attacks. WEvade achieves evasion rates of $1.00$ for Stable Signature but only $0.16$ for FeatMark in the removal setting, and $1.00$ vs. $0.14$ in the forgery setting. Black-box removal and regeneration attacks also have little effect on FeatMark: UnMarker, diffusion regeneration, VAE reconstruction, and WatermarkAttacker change FeatMark’s bit accuracy by at most about $0.02$ , while substantially degrading pixel- or coefficient-level baselines.

Video mimicry and adaptive attacks. FeatMark also extends to video mimicry, where the verifier achieves $0.85$ bit accuracy on watermarked videos versus $0.13$ on non-watermarked videos. Details are provided in Appendix D. We further evaluate mechanism-aware adaptive attacks in Appendix E.

5 Conclusion

We presented FeatMark, a feature-level watermarking framework for attributing diffusion-based mimicry attacks. FeatMark encodes provenance as inconspicuous semantic micro-features selected automatically from a feature bank and transferred through mimicry training via image cloaking. Experiments on faces, artworks, and video show near–No-Defense perceptual quality, clear watermarked/non-watermarked separation, and strong robustness to purification, removal, and tailored adaptive attacks.

References

[1] V. Asnani, J. Collomosse, T. Bui, X. Liu, and S. Agarwal (2024) ProMark: proactive diffusion watermarking for causal attribution. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 10802–10811. Cited by: Appendix F, Appendix I.
[2] E. Bagdasaryan, A. Veit, Y. Hua, D. Estrin, and V. Shmatikov (2020) How to backdoor federated learning. In International conference on artificial intelligence and statistics, pp. 2938–2948. Cited by: §1.
[3] J. Ballé, D. Minnen, S. Singh, S. J. Hwang, and N. Johnston (2018) Variational image compression with a scale hyperprior. arXiv preprint arXiv:1802.01436. Cited by: Appendix A, §1.
[4] T. Brooks, A. Holynski, and A. A. Efros (2023) Instructpix2pix: learning to follow image editing instructions. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 18392–18402. Cited by: §3.5.
[5] G. Brown, A. Pocock, M. Zhao, and M. Luján (2012) Conditional likelihood maximisation: a unifying framework for information theoretic feature selection. Journal of Machine Learning Research 13 (2), pp. 27–66. External Links: Link Cited by: §3.3, §3.4.
[6] B. Cao, C. Li, T. Wang, J. Jia, B. Li, and J. Chen (2024) IMPRESS: evaluating the resilience of imperceptible perturbations against unauthorized data usage in diffusion-based generative ai. Advances in Neural Information Processing Systems 36. Cited by: Appendix A, Appendix A, §1, §2, §3.6.
[7] H. Cao, D. G. Cooper, M. K. Keutmann, R. C. Gur, A. Nenkova, and R. Verma (2014) CREMA-d: crowd-sourced emotional multimodal actors dataset. IEEE Transactions on Affective Computing. Cited by: §D.1.
[8] Q. Cao, L. Shen, W. Xie, O. M. Parkhi, and A. Zisserman (2018) Vggface2: a dataset for recognising faces across pose and age. In 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018), pp. 67–74. Cited by: Appendix A, §1.
[9] X. Chen, X. Gao, J. Zhao, K. Ye, and C. Xu (2023) Advdiffuser: natural adversarial example synthesis with diffusion models. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 4562–4572. Cited by: §1.
[10] Coalition for Content Provenance and Authenticity (C2PA) (2025) Content credentials: c2pa technical specification, version 2.3. Note: Technical specificationVersion 2.3 (December 2025), Appendix A.7. Accessed 2026-01-14 External Links: Link Cited by: Appendix F.
[11] G. Couairon, J. Verbeek, H. Schwenk, and M. Cord (2022) DiffEdit: diffusion-based semantic image editing with mask guidance. arXiv preprint arXiv:2210.11427. Cited by: §3.5.
[12] Y. Cui, J. Ren, Y. Lin, H. Xu, P. He, Y. Xing, W. Fan, H. Liu, and J. Tang (2023) Ft-shield: a watermark against unauthorized fine-tuning in text-to-image diffusion models. arXiv preprint arXiv:2310.02401. Cited by: Appendix F, Appendix I, §1, §1.
[13] Y. Cui, J. Ren, H. Xu, P. He, H. Liu, L. Sun, Y. Xing, and J. Tang (2024) DiffusionShield: a watermark for copyright protection against generative diffusion models. External Links: 2306.04642, Link Cited by: Appendix I, §1.
[14] P. Dhariwal and A. Nichol (2021) Diffusion models beat gans on image synthesis. Advances in neural information processing systems 34, pp. 8780–8794. Cited by: §1.
[15] Z. Dong, C. Shuai, Z. Ba, P. Cheng, Z. Qin, Q. Wang, and K. Ren (2025) WMCopier: forging invisible image watermarks on arbitrary images. arXiv preprint arXiv:2503.22330. Cited by: Appendix F.
[16] (2024) Article 50: Transparency Obligations for Providers and Deployers of Certain AI Systems (regulation (eu) 2024/1689). European Parliament and Council. Note: https://artificialintelligenceact.eu/article/50/Official Journal of the European Union, L 2024/1689, 12 July 2024 Cited by: §1.
[17] W. Feng, J. He, J. Zhang, T. Zhang, W. Zhou, W. Zhang, and N. Yu (2023) Catch you everything everywhere: guarding textual inversion via concept watermarking. External Links: 2309.05940, Link Cited by: Appendix F.
[18] P. Fernandez, G. Couairon, H. Jégou, M. Douze, and T. Furon (2023) The stable signature: rooting watermarks in latent diffusion models. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 22466–22477. Cited by: Appendix A, Appendix A, Appendix F, Appendix I, §1, §3.7.
[19] H. Foerster, S. Behrouzi, P. Rieger, M. Jadliwala, and A. Sadeghi (2025) LightShed: defeating perturbation-based image copyright protections. In Proceedings of the 34th USENIX Security Symposium (USENIX Security 2025), Seattle, WA, USA. Note: Short Presentation External Links: Link Cited by: Appendix F.
[20] R. Gal, Y. Alaluf, Y. Atzmon, O. Patashnik, A. H. Bermano, G. Chechik, and D. Cohen-Or (2022) An image is worth one word: personalizing text-to-image generation using textual inversion. arXiv preprint arXiv:2208.01618. Cited by: Appendix F, §G.2, Table 10.
[21] R. Geirhos, J. Jacobsen, C. Michaelis, R. Zemel, W. Brendel, M. Bethge, and F. A. Wichmann (2020) Shortcut learning in deep neural networks. Nature Machine Intelligence 2 (11), pp. 665–673. External Links: Document Cited by: §3.3.
[22] M. Guo, Y. Hu, Z. Jiang, Z. Li, A. Sadovnik, A. Daw, and N. Gong (2024) AI-generated image detection: passive or watermark?. External Links: 2411.13553, Link Cited by: Appendix F, §1.
[23] I. Guyon and A. Elisseeff (2003) An introduction to variable and feature selection. Journal of Machine Learning Research 3, pp. 1157–1182. Cited by: §3.3, §3.4.
[24] D. Hendrycks, K. Zhao, S. Basart, J. Steinhardt, and D. Song (2021) Natural adversarial examples. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 15262–15271. Cited by: §1.
[25] A. Hertz, R. Mokady, J. Tenenbaum, K. Aberman, Y. Pritch, and D. Cohen-Or (2022) Prompt-to-prompt image editing with cross attention control. arXiv preprint arXiv:2208.01626. External Links: Link Cited by: §3.5.
[26] R. Hönig, J. Rando, N. Carlini, and F. Tramèr (2024) Adversarial perturbations cannot reliably protect artists from generative ai. arXiv preprint arXiv:2406.12027. Cited by: Appendix A, Appendix F, §1, §3.6.
[27] E. J. Hu, Y. Shen, P. Wallis, Z. Allen-Zhu, Y. Li, S. Wang, L. Wang, and W. Chen (2021) LoRA: low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685. Cited by: §D.1, Appendix F, §G.2, Table 10.
[28] Y. Hu, Z. Jiang, M. Guo, and N. Gong (2024) A transfer attack to image watermarks. arXiv preprint arXiv:2403.15365. Cited by: Appendix F, §1.
[29] Y. Hu, Z. Jiang, M. Guo, and N. Gong (2024) Stable signature is unstable: removing image watermark from diffusion models. arXiv preprint arXiv:2405.07145. Cited by: Appendix F, §1.
[30] L. Huang, W. Yu, W. Ma, W. Zhong, Z. Feng, H. Wang, Q. Chen, W. Peng, X. Feng, B. Qin, et al. (2023) A survey on hallucination in large language models: principles, taxonomy, challenges, and open questions. arXiv preprint arXiv:2311.05232. Cited by: §1.
[31] International Telecommunication Union (2016) Recommendation itu-t p.915: subjective test methodologies for 3d video quality assessment. Technical report International Telecommunication Union. Note: Section on two-alternative forced choice pair comparison (2AFC-PC) Cited by: Appendix A.
[32] International Telecommunication Union (2019) Recommendation itu-r bt.500-14: methodologies for the subjective assessment of the quality of television pictures. Technical report International Telecommunication Union. Note: Available from the ITU website Cited by: Appendix A.
[33] Z. Jiang, J. Zhang, and N. Z. Gong (2023) Evading watermark based detection of ai-generated content. In Proceedings of the 2023 ACM SIGSAC Conference on Computer and Communications Security, pp. 1168–1181. Cited by: Appendix A, Appendix F, Appendix I, §1, §1.
[34] Jr. Joseph R. Biden (2023) Executive Order 14110: Safe, Secure, and Trustworthy Development and Use of Artificial Intelligence. Note: https://www.federalregister.gov/documents/2023/11/01/2023-24283/safe-secure-and-trustworthy-development-and-use-of-artificial-intelligence88 Fed. Reg. 75191 (Nov. 1, 2023) Cited by: §1.
[35] T. Karras, T. Aila, S. Laine, and J. Lehtinen (2018) Progressive growing of gans for improved quality, stability, and variation. In International Conference on Learning Representations, Cited by: Appendix A, §1.
[36] T. Karras, S. Laine, and T. Aila (2019) A style-based generator architecture for generative adversarial networks. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 4401–4410. Cited by: §1.
[37] A. Kassis and U. Hengartner (2024) UnMarker: a universal attack on defensive watermarking. arXiv preprint arXiv:2405.08363. Cited by: Appendix A, Appendix F, Appendix I, §1.
[38] E. Kenneally and D. Dittrich (2012) The menlo report: ethical principles guiding information and communication technology research. Note: U.S. Department of Homeland Security External Links: Link Cited by: Ethical Considerations.
[39] A. Kirillov, E. Mintun, N. Ravi, H. Mao, C. Rolland, L. Gustafson, T. Xiao, S. Whitehead, A. C. Berg, W. Lo, P. Dollár, and R. Girshick (2023) Segment anything. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), Cited by: Appendix A, §E.1.
[40] N. Kumari, B. Zhang, R. Zhang, E. Shechtman, and J. Zhu (2023) Multi-concept customization of text-to-image diffusion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1931–1941. Cited by: Appendix F.
[41] T. Kynkäänniemi, T. Karras, S. Laine, J. Lehtinen, and T. Aila (2019) Improved precision and recall metric for assessing generative models. Advances in neural information processing systems 32. Cited by: Appendix A.
[42] G. Li, Y. Chen, J. Zhang, J. Li, S. Guo, and T. Zhang (2024) Warfare:breaking the watermark protection of ai-generated content. External Links: 2310.07726, Link Cited by: Appendix F.
[43] Y. Li, W. Zhang, X. Lyu, Y. Liu, and B. Xiao (2025) StyleGuard: preventing Text-to-Image-model-based style mimicry attacks by style perturbations. In The Thirty-ninth Annual Conference on Neural Information Processing Systems, External Links: Link Cited by: Appendix F.
[44] C. Liang, X. Wu, Y. Hua, J. Zhang, Y. Xue, T. Song, Z. Xue, R. Ma, and H. Guan (2023) Adversarial example does good: preventing painting imitation from diffusion models via adversarial examples. arXiv preprint arXiv:2302.04578. Cited by: Appendix F, Appendix I, §3.1.
[45] J. Lin and M. Juarez (2025) A crack in the bark: leveraging public knowledge to remove tree-ring watermarks. arXiv preprint arXiv:2506.10502. Cited by: Appendix F.
[46] H. Liu, M. Guo, Z. Jiang, L. Wang, and N. Z. Gong (2024) AudioMarkBench: benchmarking robustness of audio watermarking. arXiv preprint arXiv:2406.06979. Cited by: §1.
[47] S. Liu, Z. Zeng, T. Ren, F. Li, H. Zhang, J. Yang, Q. Jiang, C. Li, J. Yang, H. Su, J. Zhu, and L. Zhang (2024) Grounding dino: marrying dino with grounded pre-training for open-set object detection. In European Conference on Computer Vision (ECCV), Cited by: Appendix A, §E.1.
[48] Y. Liu, C. Fan, Y. Dai, X. Chen, P. Zhou, and L. Sun (2024) MetaCloak: preventing unauthorized subject-driven text-to-image diffusion-based synthesis via meta-learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 24219–24228. Cited by: Appendix A, Appendix F.
[49] Z. Liu, P. Luo, X. Wang, and X. Tang (2015) Deep learning face attributes in the wild. In Proceedings of the IEEE international conference on computer vision, pp. 3730–3738. Cited by: Appendix A.
[50] I. Loshchilov F. Hutter et al. (2017) Fixing weight decay regularization in adam. arXiv preprint arXiv:1711.05101 5. Cited by: Appendix A.
[51] N. Lukas, A. Diaa, L. Fenaux, and F. Kerschbaum (2023) Leveraging optimization for adaptive attacks on image watermarks. arXiv preprint arXiv:2309.16952. Cited by: Appendix F.
[52] Y. Ma, Z. Zhao, X. He, Z. Li, M. Backes, and Y. Zhang (2023) Generative watermarking against unauthorized subject-driven image synthesis. External Links: 2306.07754, Link Cited by: Appendix F, Appendix I.
[53] R. K. Mantiuk, K. J. Kim, A. G. Rempel, and W. Heidrich (2012) Comparison of four subjective methods for image quality assessment. Computer Graphics Forum 31 (8), pp. 2478–2491. Cited by: Appendix A.
[54] C. Meng, Y. He, Y. Song, J. Song, J. Wu, J. Zhu, and S. Ermon (2022) SDEdit: guided image synthesis and editing with stochastic differential equations. In International Conference on Learning Representations (ICLR), External Links: Link, 2108.01073 Cited by: §E.1, §3.5.
[55] R. Min, S. Li, H. Chen, and M. Cheng (2024) A watermark-conditioned diffusion model for ip protection. External Links: 2403.10893, Link Cited by: Appendix F.
[56] T. Moon, M. Choi, G. Lee, J. Ha, and J. Lee (2022) Fine-tuning diffusion models with limited data. In NeurIPS 2022 Workshop on Score-Based Methods, Cited by: §1.
[57] A. Mustafa, S. H. Khan, M. Hayat, J. Shen, and L. Shao (2019) Image super-resolution as a defense against adversarial attacks. IEEE Transactions on Image Processing 29, pp. 1711–1724. Cited by: Appendix A.
[58] M. F. Naeem, S. J. Oh, Y. Uh, Y. Choi, and J. Yoo (2020) Reliable fidelity and diversity metrics for generative models. In International Conference on Machine Learning, pp. 7176–7185. Cited by: Appendix A.
[59] W. Nie, B. Guo, Y. Huang, C. Xiao, A. Vahdat, and A. Anandkumar (2022) Diffusion models for adversarial purification. arXiv preprint arXiv:2205.07460. Cited by: Appendix A, §1.
[60] H. Peng, F. Long, and C. Ding (2005) Feature selection based on mutual information criteria of max-dependency, max-relevance, and min-redundancy. IEEE Transactions on Pattern Analysis and Machine Intelligence 27 (8), pp. 1226–1238. External Links: Document Cited by: §3.3, §3.4.
[61] Prolific (2024) Prolific | quickly find research participants you can trust. External Links: Link Cited by: Appendix A.
[62] A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark, et al. (2021) Learning transferable visual models from natural language supervision. In International conference on machine learning, pp. 8748–8763. Cited by: Appendix A, §3.3, §3.3, §3.7.
[63] A. Ramesh, M. Pavlov, G. Goh, S. Gray, C. Voss, A. Radford, M. Chen, and I. Sutskever (2021) Zero-shot text-to-image generation. In International conference on machine learning, pp. 8821–8831. Cited by: §1.
[64] M. S. Rana, M. N. Nobi, B. Murali, and A. H. Sung (2022) Deepfake detection: a systematic literature review. IEEE access 10, pp. 25494–25513. Cited by: §1.
[65] N. Ravi et al. (2024) SAM 2: segment anything in images and videos. arXiv preprint arXiv:2408.00714. Cited by: §D.1.
[66] T. Ren, S. Liu, A. Zeng, J. Lin, K. Li, H. Cao, J. Chen, X. Huang, Y. Chen, F. Yan, Z. Zeng, H. Zhang, F. Li, J. Yang, H. Li, Q. Jiang, and L. Zhang (2024) Grounded sam: assembling open-world models for diverse visual tasks. arXiv preprint arXiv:2401.14159. Cited by: Appendix A, §E.1.
[67] R. Rombach, A. Blattmann, D. Lorenz, P. Esser, and B. Ommer (2022) High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 10684–10695. Cited by: Appendix A, §1.
[68] N. Ruiz, Y. Li, V. Jampani, Y. Pritch, M. Rubinstein, and K. Aberman (2023) Dreambooth: fine tuning text-to-image diffusion models for subject-driven generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 22500–22510. Cited by: Appendix A, Appendix A, Appendix F, Table 10, §1, §2.
[69] V. S. Sadasivan, A. Kumar, S. Balasubramanian, W. Wang, and S. Feizi (2023) Can ai-generated text be reliably detected?. arXiv preprint arXiv:2303.11156. Cited by: Appendix I.
[70] B. Saleh and A. Elgammal (2015) Large-scale classification of fine-art paintings: learning the right metric on the right feature. arXiv preprint arXiv:1505.00855. Cited by: Appendix A, §1.
[71] H. Salman, A. Khaddaj, G. Leclerc, A. Ilyas, and A. Madry (2023) Raising the cost of malicious ai-powered image editing. arXiv preprint arXiv:2302.06588. Cited by: Appendix F, §3.1, §3.2, §3.6.
[72] S. Shan, J. Cryan, E. Wenger, H. Zheng, R. Hanocka, and B. Y. Zhao (2023) Glaze: protecting artists from style mimicry by $\{$ text-to-image $\}$ models. In 32nd USENIX Security Symposium (USENIX Security 23), pp. 2187–2204. Cited by: Appendix A, Appendix A, Appendix F, §2, §3.1, §3.2, §3.6.
[73] S. Shan, W. Ding, J. Passananti, S. Wu, H. Zheng, and B. Y. Zhao (2024) Nightshade: prompt-specific poisoning attacks on text-to-image generative models. In 2024 IEEE Symposium on Security and Privacy (SP), pp. 212–212. Cited by: Appendix F.
[74] B. Sun, J. Sun, W. Koh, and J. Shi (2024) Neural network semantic backdoor detection and mitigation: a $\{$ causality-based $\}$ approach. In 33rd USENIX Security Symposium (USENIX Security 24), pp. 2883–2900. Cited by: §1.
[75] M. Tancik, B. Mildenhall, and R. Ng (2020) StegaStamp: invisible hyperlinks in physical photographs. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2117–2126. Cited by: Appendix A, Appendix F.
[76] Q. Tang, J. Krinsky, and A. Bharati (2025) StyleProtect: safeguarding artistic identity in fine-tuned diffusion models. arXiv preprint arXiv:2509.13711. External Links: Link Cited by: Appendix F.
[77] T. Van Le, H. Phung, T. H. Nguyen, Q. Dao, N. N. Tran, and A. Tran (2023) Anti-dreambooth: protecting users from personalized text-to-image synthesis. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 2116–2127. Cited by: Appendix A, Appendix A, Appendix F, §3.1.
[78] B. Wang, Y. Yao, S. Shan, H. Li, B. Viswanath, H. Zheng, and B. Y. Zhao (2019) Neural Cleanse: identifying and mitigating backdoor attacks in neural networks. In 2019 IEEE Symposium on Security and Privacy (SP), pp. 707–723. Cited by: Appendix F, §G.5.
[79] J. Wang, X. Liu, J. Hu, D. Wang, S. Wu, T. Jiang, Y. Guo, A. Liu, and J. Zhou (2023) Adversarial examples in the physical world: a survey. arXiv preprint arXiv:2311.01473. Cited by: §1.
[80] R. Wang, H. Chen, Z. Zhu, L. Liu, and B. Wu (2023) Versatile backdoor attack with visible, semantic, sample-specific, and compatible triggers. arXiv preprint arXiv:2306.00816. Cited by: §1.
[81] Z. Wang, C. Chen, L. Lyu, D. N. Metaxas, and S. Ma (2023) Diagnosis: detecting unauthorized data usages in text-to-image diffusion models. In The Twelfth International Conference on Learning Representations, Cited by: Appendix F.
[82] Y. Wen, J. Kirchenbauer, J. Geiping, and T. Goldstein (2023) Tree-ring watermarks: fingerprints for diffusion images that are invisible and robust. arXiv preprint arXiv:2305.20030. Cited by: Appendix A, Appendix F, §1, §1.
[83] E. Xie, L. Yao, H. Shi, Z. Liu, D. Zhou, Z. Liu, J. Li, and Z. Li (2023) Difffit: unlocking transferability of large diffusion models via simple parameter-efficient fine-tuning. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 4230–4239. Cited by: §1.
[84] H. Xue, C. Liang, X. Wu, and Y. Chen (2023) Toward effective protection against diffusion-based mimicry through score distillation. In The Twelfth International Conference on Learning Representations, Cited by: Appendix F.
[85] Z. Yang, J. Teng, W. Zheng, M. Ding, S. Huang, J. Xu, Y. Yang, W. Hong, X. Zhang, G. Feng, D. Yin, X. Gu, Y. Zhang, W. Wang, Y. Cheng, T. Liu, B. Xu, Y. Dong, and J. Tang (2024) CogVideoX: text-to-video diffusion models with an expert transformer. arXiv preprint arXiv:2408.06072. Cited by: §D.1.
[86] Z. Yang, K. Zeng, K. Chen, H. Fang, W. Zhang, and N. Yu (2024) Gaussian shading: provable performance-lossless image watermarking for diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 12162–12171. Cited by: Appendix F, Appendix I.
[87] H. Ye, J. Zhang, S. Liu, X. Han, and W. Yang (2023) IP-Adapter: text compatible image prompt adapter for text-to-image diffusion models. arXiv preprint arXiv:2308.06721. External Links: Link Cited by: Appendix F.
[88] Y. Yu, W. Zhang, and Y. Deng (2021) Frechet inception distance (fid) for evaluating gans. China University of Mining Technology Beijing Graduate School 3. Cited by: Appendix A.
[89] H. Zhang, B. L. Edelman, D. Francati, D. Venturi, G. Ateniese, and B. Barak (2024) Watermarks in the sand: impossibility of strong watermarking for language models. In Forty-first International Conference on Machine Learning, Cited by: Appendix I, §1.
[90] L. Zhang, X. Liu, A. V. Martin, C. X. Bearfield, Y. Brun, and H. Guan (2024) Robust image watermarking using stable diffusion. External Links: 2401.04247, Link Cited by: Appendix F, Appendix I.
[91] X. Zhang, R. Li, J. Yu, Y. Xu, W. Li, and J. Zhang (2024) EditGuard: versatile image watermarking for tamper localization and copyright protection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 11964–11974. Cited by: Appendix F.
[92] X. Zhao, K. Zhang, Z. Su, S. Vasan, I. Grishchenko, C. Kruegel, G. Vigna, Y. Wang, and L. Li (2024) Invisible image watermarks are provably removable using generative ai. Advances in neural information processing systems 37, pp. 8643–8672. Cited by: Appendix A, Appendix A, Appendix F, §1.
[93] K. Zhou, J. Yang, C. C. Loy, and Z. Liu (2022) Conditional prompt learning for vision-language models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 16816–16825. External Links: Document Cited by: §3.3.
[94] J. Zhu, R. Kaplan, J. Johnson, and L. Fei-Fei (2018) Hidden: hiding data with deep networks. In Proceedings of the European conference on computer vision (ECCV), pp. 657–672. Cited by: Appendix A, Appendix A, Appendix F, §1, §3.7.
[95] P. Zhu, T. Takahashi, and H. Kataoka (2024) Watermark-embedded adversarial examples for copyright protection against diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 24420–24430. Cited by: Appendix I.

Appendix

Appendix A Full Experimental Setup

We evaluate FeatMark from three perspectives: (1) watermark detection accuracy on attacker-generated images, (2) perceptual quality of the generated outputs, and (3) robustness of watermark verification under purification and watermark-removal attacks. In every experiment, the attacker receives one of three training sets, original images (No-Defense), images protected by a baseline watermark/defense, or FeatMark-protected released images, and performs mimicry fine-tuning to generate fake images. We then evaluate those fakes using the metrics below.

Datasets. We follow the dataset protocols used in prior cloaking and purification evaluations [77, 72, 6]. We consider two face datasets (VGGFace2 [8] and CelebA-HQ [35, 49]) and one artwork dataset (WikiArt [70]). We use the standard partitions and mimicry settings from DreamBooth-style evaluations [68, 77, 6]. Since VGGFace2 has higher identity/attribute diversity, we additionally use it for adaptive-attack robustness and ablations [77, 8].

Implementations. We instantiate the open-vocabulary evidence scorer with CLIP [62]. For region grounding $M_{k}(x)$ we use a text-guided open-world segmentation pipeline (LangSAM) built upon GroundingDINO [47] and Segment Anything [39] via Grounded-SAM [66]. For the concept-guided editor $\mathcal{E}$ in Eq. (6), we use a diffusion-based image-to-image inpainting pipeline conditioned on the edit instruction and mask; other instruction-following masked editors can be substituted without changing the methodology.

Training configuration. We optimize each image cloak for 500 iterations with step size $1.0/255$ , $\lambda=10^{2}$ in Eq. (8), and default perturbation budget $0.10$ (ablations in Appendix H). Mimicry attacks use DreamBooth [68] for 1000 steps on a Stable Diffusion v2.1 base [67] (additional diffusion versions and personalization backends are evaluated in Appendices G.1 and G.2). The watermark extractor is trained for 100 iterations with AdamW ( $lr=10^{-2}$ , $weight\_decay=10^{-2}$ ); we use 48-bit messages and follow HiDDeN [94] to train the output layer with AdamW ( $lr=0.01$ , $weight\_decay=0.01$ ) [50]. Unless otherwise stated, attribution is evaluated by bit accuracy and a fixed decision threshold chosen on a held-out calibration split; all WM=Y/WM=N tables report the resulting separation rather than a formal proof of verification.

Compute resources. All experiments were conducted on NVIDIA A100X GPUs with 80GB memory, using NVIDIA driver 550.54.14 and CUDA 12.4. A DreamBooth mimicry run with 1,000 fine-tuning steps took about 10 minutes on one A100X GPU. LoRA and Textual Inversion runs were lighter and took approximately 3–6 minutes per target. The remaining stages, including concept selection, localized editing, image cloaking, reader training, watermark extraction, purification/removal attacks, and adaptive attack evaluation, were run on the same GPU type. The complete experimental suite consumed approximately 150–250 A100X GPU-hours in total.

Evaluation metrics. Watermark accuracy is measured by bit accuracy (fraction of correctly decoded bits). Unless stated otherwise, we report robustness under common transformations [18]: strong crop (retain 90%), brightness $\times 2.0$ , contrast $\times 2.0$ , JPEG quality 50, and a combined transform (Crop+Brigh.+JPEG). Perceptual quality is evaluated on the attacker-generated images using FID [88], precision/recall [41], and coverage/density [58] between generated fakes and the original images.

Removal and purification attacks. To stress-test robustness, we evaluate purification attacks applied before mimicry training (IMPRESS [6] and Noisy Upscaling [26, 57]), a white-box watermark remover with extractor access (WEvade [33]), and black-box removal/regeneration attacks including UnMarker [37], diffusion-based regeneration [59], VAE reconstruction [3], and the WatermarkAttacker framework with multiple configurations [92].

Baselines. We compare against representative invisible watermark baselines for diffusion models (Stable Signature [18] and Tree-Ring [82]). We additionally include learned message watermarking baselines HiDDeN [94] and StegaStamp [75]. Since prior image cloaks are not designed for verification, we also include Anti-DreamBooth [77], and for WikiArt we include Glaze [72], to contextualize perceptual quality. We further add MetaCloak [48], a meta-learning-based poisoning defense designed to impede subject-driven diffusion fine-tuning. For removal-attack comparisons, we additionally report classic frequency-/learning-based schemes (DwtDct, DwtDctSvd, RivaGAN) as used by WatermarkAttacker [92].

User study. We conduct a pairwise study comparing attacker-generated images from No-Defense versus FeatMark, using the same target and prompt per pair. Order is randomized and balanced. Participants answer which image is more realistic, which better matches the target identity/style, and which better matches the edit instruction, following standard pair-comparison guidance [32, 31, 53]. We recruit 49 participants on Prolific [61] to label 100 image pairs (18+, no special face-recognition expertise), with a 30-minute limit and £3.00 compensation.

Appendix B Additional Watermark Accuracy Discussion

A key advantage of FeatMark is the verification margin between WM=Y and WM=N, which directly translates into lower false positives and more reliable attribution. For VGGFace2 and WikiArt, FeatMark drives WM=N accuracy close to zero across all transformations (e.g., 0.03 to 0.05 on VGGFace2 and 0.01 to 0.02 on WikiArt), while keeping WM=Y accuracy high. Even under cropping, FeatMark preserves strong separability, with 0.75 vs. 0.04 on VGGFace2 and 0.26 vs. 0.02 on WikiArt. In contrast, Stable Signature exhibits WM=N accuracies near random guessing around 0.45–0.56, and most critically its separability collapses under cropping where WM=Y becomes comparable to WM=N (e.g., 0.36 vs. 0.40 on VGGFace2, 0.38 vs. 0.42 on CelebA-HQ, and 0.50 vs. 0.56 on WikiArt). While all these methods can decode well in benign conditions, FeatMark more consistently maintains a usable decision margin when the attacker applies realistic removal attack. Cropping is inherently the most challenging transformation for localized semantic watermarks, since it can remove the edited region entirely. Nevertheless, FeatMark’s design still yields a practical attribution signal whenever part of the marked region remains, and its strong suppression of WM=N decoding provides a cleaner and more reliable verification behavior than baselines across datasets and perturbations.

Appendix C Additional Qualitative Results

The CelebA-HQ examples complement the VGGFace2 and WikiArt visualizations in the main paper. We also provide the Noisy Upscaling visualization for the purification results discussed in Section 4.4.

Appendix D Video Mimicry Extension

D.1 Extensibility to Video Mimicry

We extend FeatMark from images to the video mimicry setting by replacing the image editor with a masked video-to-video editor. Concretely, for each target video we first select one subtle feature from our feature bank, then localize it on the first frame via text-guided segmentation and propagate the mask across the entire clip using a promptable video segmentation tracker (SAM 2) [65]. Given the propagated mask sequence, we apply a single concept instruction consistently to all frames via a masked video editing backbone, yielding an edited video whose watermark evidence is temporally coherent.We then adapt our cloaking stage from a single image to a video by distributing the cloaking perturbation across frames. Specifically, we add bounded, per-frame noise after editing so that the published video remains visually faithful while still steering the downstream mimicry process toward reproducing the injected feature evidence throughout the clip.

On the attacker side, we instantiate video mimicry using CogVideoX [85] and perform LoRA-based [27] fine-tuning on the target character videos from CREMA-D [7]. Evaluating the resulting fake videos, our verifier achieves a bit accuracy of $0.13$ on non-watermarked samples and $0.85$ on watermarked samples, indicating a clear separation between clean and FeatMark-protected content in the video domain.

Appendix E Adaptive Attack Details

The main paper summarizes the adaptive attacks and keeps the full quantitative table and qualitative visualization here.

Table 8: Bit accuracy change of FeatMark against tailored adaptive attacks on VGGFace2.

Attack Change of Bit Accuracy None Crop Brigh. Cont. JPEG Comb. TCN $-$ 0.02 $-$ 0.00 $-$ 0.07 $-$ 0.01 $-$ 0.02 $-$ 0.09 AFT $-$ 0.01 $-$ 0.00 $-$ 0.03 $-$ 0.01 $-$ 0.02 $-$ 0.01 DRA-G $-$ 0.20 $-$ 0.02 $-$ 0.22 $-$ 0.26 $-$ 0.20 $-$ 0.23 DRA-G $-$ 0.22 $-$ 0.07 $-$ 0.25 $-$ 0.27 $-$ 0.24 $-$ 0.32

E.1 Attack Methodology

We evaluate three adaptive attackers with increasing capability, and refer to them by short names in later results.

TCN (Targeted Concept Neutralization). FeatMark’s watermark is realized as a localized concept, so a natural adaptive strategy is to infer the concept and undo it. TCN ranks candidate bank features using open-vocabulary concept evidence, localizes the predicted feature region with text-guided grounding, and applies a counterfactual edit that neutralizes the attribute while preserving identity and global semantics. We instantiate grounding with open-world detection and segmentation tools commonly assembled as Grounded-SAM [47, 39, 66]. This attack is practical because it uses off-the-shelf grounding and editing modules and is directly aligned with the same “detect–ground–edit” abstraction as our concept programs, making it a targeted stress test for feature-anchored watermarks.

AFT (Adversarial Fine-Tuning). A stronger attacker can push watermark evasion into training, so that the generator learns to avoid verifiable outputs by construction. Assuming white-box access to the reader $\mathcal{R}$ and message $m$ , the attacker fine-tunes mimicry model parameters $\theta$ to suppress decoded bits while preserving identity or style relative to a reference checkpoint $\theta_{0}$ . Using a deterministic sampling routine $x_{\theta}(z,c)$ from the mimicry model for noise seed $z$ and condition $c$ , a representative objective is

	$\displaystyle\min_{\theta}$	$\displaystyle\mathbb{E}_{z,c}\Big[\mathrm{BCE}\big(m,\mathcal{R}(x_{\theta}(z,c))\big)\Big]$		(10)
	$\displaystyle\text{s.t.}$	$\displaystyle\mathrm{Sim}\big(x_{\theta}(z,c),x_{\theta_{0}}(z,c)\big)\geq\gamma.$		(10)

As such, AFT provides a meaningful upper bound because it directly optimizes against the exact verification mechanism under an explicit fidelity constraint.

DRA (Diffusion Regeneration Attack). DRA models a practical “regenerate-to-remove” threat where an attacker applies diffusion-based image-to-image regeneration to rewrite details while keeping global content largely intact, as in SDEdit-style editing and diffusion purification [54]. We evaluate two variants. DRA-UG (unguided) simply regenerates each image with a tunable strength (e.g., noise level or diffusion steps) and selects the strongest setting that stays within a perceptual similarity budget. DRA-G (guided) additionally uses the watermark reader as feedback, choosing regeneration settings—and optionally adding reader-guidance during regeneration—to directly minimize verification, e.g., by reducing $\mathcal{B}_{\mathrm{acc}}\!\left(m,\mathcal{R}(\tilde{x})\right)$ (or an equivalent differentiable surrogate such as $\mathrm{BCE}(m,\mathcal{R}(\tilde{x}))$ ). Together, DRA-UG and DRA-G capture both an off-the-shelf post-processing attacker and a stronger mechanism-aware variant that explicitly targets the decoded bits.

Appendix F Related Work

We position FeatMark against four lines of work: personalization attacks, watermark-based attribution, anti-mimicry cloaks, and watermark/cloak removal attacks. This organization is important because many defenses that look superficially comparable optimize for a different security goal.

Personalization and mimicry attacks. Modern mimicry pipelines personalize text-to-image generators from a small reference set. DreamBooth binds a rare token to a subject or style through fine-tuning [68], Textual Inversion learns a new textual embedding while keeping the generator mostly fixed [20], and parameter-efficient variants such as LoRA update low-rank adapters rather than all model weights [27]. Other personalization interfaces, such as Custom Diffusion [40] and IP-Adapter [87], further broaden the attacker design space by changing how image identity/style is injected. FeatMark is not tied to a single personalization implementation: its signal is placed in localized semantic evidence that a successful mimicry model must reproduce to imitate the target. Appendix G.2 therefore evaluates LoRA and Textual Inversion in addition to the DreamBooth protocol used in the main experiments.

Watermarks and provenance. Generation-time watermarks integrate provenance into the synthesis process. Stable Signature [18], Tree-Ring [82], Gaussian Shading [86], ZoDiac [90], WaDiff [55], EditGuard [91], and WMCopier [15] embed or propagate low-level signals that can be decoded later; earlier image watermarking systems such as HiDDeN [94] and StegaStamp [75] provide learned message decoders for natural images. Source-side residue methods place an imperceptible mark on the public training images before release, including ProMark [1], GenWatermark [52], DIAGNOSIS [81], FT-Shield [12], and related concept-watermarking schemes for textual inversion [17]. Metadata standards such as C2PA [10] are complementary but do not survive the training process once an adversary extracts raw pixels. FeatMark differs in mechanism and threat model: it makes the provenance signal part of the visual/semantic data distribution that the mimicry model learns, rather than relying on a low-energy pixel signature or a file-container credential.

Anti-mimicry cloaks and style-protection defenses. Cloaking defenses add adversarial perturbations to public images so that personalization learns the wrong identity/style. Representative methods include AdvDM [44], PhotoGuard [71], Anti-DreamBooth [77], Glaze [72], MetaCloak [48], Nightshade [73], LightShed [19], and related image-cloak analyses [84, 26]. Recent style-specific defenses such as StyleGuard [43] and StyleProtect [76] strengthen perturbation-based style protection by targeting style-related features or selected diffusion components. These works are important baselines for disruption: they try to make the attacker fail to reproduce the original identity/style. FeatMark instead targets attribution: it intentionally preserves usable mimicry quality while causing successful mimicry outputs to carry a decodable semantic trace. This distinction explains why disruption metrics alone are insufficient for evaluating FeatMark and why our main tables report WM=Y/WM=N separability, bit accuracy, and perceptual quality rather than only attack failure rates.

Removal, purification, and adaptive trigger-inference attacks. Pixel-level watermarks and cloaks can be attacked through adversarial perturbation, regeneration, or spectral manipulation. WEvade [33, 22], surrogate-key optimization [51], watermark forgery [42], unstable diffusion-component fine-tuning [29], transfer attacks [28], UnMarker [37], WatermarkAttacker [92], and latent VAE attacks [45] demonstrate that invisible or coefficient-level signals are often fragile. Reviewers may also ask whether trigger-recovery methods such as Neural Cleanse [78], originally designed for backdoor triggers, can reverse-engineer FeatMark. We address this in Appendix G.5: FeatMark is not a single universal patch or fixed trigger. The selected cue, localization, edit direction/strength, and cloaking objective are sample-conditioned and defender-side, so trigger inversion can at best infer a coarse semantic hypothesis; removing the transferred evidence generally requires visible identity/style edits rather than benign denoising.

Declaration of LLM Usage

A generative AI assistant was used only for grammar and light style polishing. It was not used as an original, non-standard, or decision-making component of the research method, experiments, figures, or analysis.

Ethical Considerations

This work aims to mitigate harms from diffusion-based mimicry attacks, which can violate personal autonomy, privacy, and reputation by enabling convincing forgeries of an individual’s identity or an artist’s style. FeatMark is designed as a defensive, post-hoc attribution mechanism. Rather than improving synthesis quality or facilitating impersonation, it enables reliable verification of whether suspicious content is likely produced by a mimicry-trained model.

We follow the Menlo Report principles for ICT research [38]. Our primary stakeholders include affected individuals and artists whose publicly shared images can be misused for impersonation, researchers and practitioners who build or audit generative systems, and companies/platforms that host or deploy personalization tools. The anticipated benefits are improved provenance and accountability for misuse. Potential risks include that parts of our evaluation pipeline (e.g., mimicry fine-tuning scripts) could be repurposed to lower the barrier for abuse. The implementation and attack configurations are not publicly released. We also do not release any additional identity-specific data beyond what is already publicly available in the referenced datasets, and our verifier parameters remain private by design.

All experiments are conducted offline on publicly available datasets (VGGFace2, CelebA-HQ, WikiArt) following prior protocols, and no live systems, services, or user accounts are targeted or affected. Our user-perception study was reviewed by the authors’ institutional Human Research Ethics Committee (IRB) and determined to be exempt (minimal risk). Participants provided informed consent, were told the study purpose at a high level, and no personally identifying information was collected; responses were used solely for academic research.

Overall, we believe the societal benefit of enabling attribution and accountability for mimicry-generated forgeries outweighs the residual misuse risk, and we explicitly incorporate mitigations through stakeholder analysis and restrictions on artifact distribution.

Appendix G Additional Generalization and Deployment Evidence

G.1 Generalization across Diffusion Models

Table 9 evaluates transfer when the attacker uses a different diffusion backbone from the defender. We keep the watermarking model at Stable Diffusion v2.1 and vary the mimicry model across v2.0, v1.5, and v1.4. Without transformations, the corresponding WM=Y/WM=N bit accuracies are $0.97/0.07$ , $0.96/0.05$ , and $0.98/0.04$ , respectively, compared with $0.98/0.03$ for the default v2.1. These results preserve a large verification margin across the tested backbones, supporting attribution without requiring the attacker to use the defender’s model version.

Table 9: Bit accuracy of FeatMark when mimicry attacks are conducted with different versions of diffusion model. v2.1 is the default setting for our watermarks and mimicry attacks (*).

Model version WM? Bit Accuracy None Crop Brigh. Cont. JPEG Comb. v2.1* Y 0.98 0.75 0.96 0.96 0.99 0.98 N 0.03 0.04 0.04 0.04 0.04 0.05 v2.0 Y 0.97 0.72 0.94 0.97 0.98 0.99 N 0.07 0.03 0.09 0.13 0.11 0.15 v1.5 Y 0.96 0.70 0.96 0.96 0.97 0.97 N 0.05 0.02 0.07 0.07 0.08 0.07 v1.4 Y 0.98 0.67 0.96 0.98 0.98 0.98 N 0.04 0.02 0.06 0.07 0.08 0.07

G.2 Transfer beyond DreamBooth

To test transfer beyond the main DreamBooth protocol, we evaluate LoRA-based fine-tuning [27] and Textual Inversion [20] on VGGFace2. Table 10 reports WM=Y/WM=N bit accuracies of $0.99/0.06$ for LoRA and $0.96/0.06$ for Textual Inversion; LoRA yields FID $185.90$ . Both backends preserve a clear verification margin, suggesting that the semantic trace can transfer across personalization methods rather than depending solely on DreamBooth fine-tuning.

Table 10: Watermark transfer under different personalization backends on VGGFace2. DreamBooth is the main-paper default.

Mimicry backend WM=Y bit acc. WM=N bit acc. FID $\downarrow$ DreamBooth [68] 0.98 0.03 180.98 LoRA [27] 0.99 0.06 185.90 Textual Inversion [20] 0.96 0.06 –

G.3 Multi-round adaptive generation and cherry-picking

A stronger attacker may repeatedly generate candidates and publish only samples that minimize the watermark reader score while staying visually close to the target. We evaluate a multi-round generate-and-filter attacker: in each round, the attacker generates four candidates, retains the one with the lowest reader score under an identity-similarity budget of $0.92$ (with the current image also retained as a candidate), and repeats this process for three rounds. Table 11 shows that FeatMark remains separable after this attack: WM=Y bit accuracy drops from $0.98$ to $0.72$ , while WM=N remains $0.03$ . The fidelity cost is also measurable, with FID increasing from $171.81$ to $192.13$ , indicating that suppressing the semantic trace is not a free post-processing operation.

Table 11: Multi-round generate-and-filter attack on VGGFace2. The attacker performs three rounds, samples four candidates per round, and keeps the lowest-reader-score candidate under an identity-similarity budget of 0.92.

Setting WM=Y bit acc. WM=N bit acc. FID $\downarrow$ Default FeatMark 0.98 0.03 171.81 Generate-and-filter attacker 0.72 0.03 192.13

G.4 Incremental real-world publishing

In real deployments, a user may not have all future images available when first configuring a watermark. We therefore evaluate an incremental setting: FeatMark selects the message and feature program from an initial seed set of only two images and then applies the same program to later released images without re-selecting the feature. Table 12 shows that this realistic variant remains close to the all-images-known setting, with WM=Y/WM=N bit accuracy of $0.98/0.04$ compared with $0.98/0.03$ by default. This suggests that feature selection does not require seeing every future release, provided that the selected semantic cue remains editable and identity/style-consistent for later images.

Table 12: Incremental publishing on VGGFace2. The seed-only setting selects the watermark program from two initial images and applies it to later releases.

Setting WM=Y bit acc. WM=N bit acc. All images available at setup 0.98 0.03 Two-image seed set, later releases added 0.98 0.04

G.5 Trigger-inversion, feature leakage, and forward security

FeatMark should not be interpreted as a classic backdoor with a single universal trigger. Backdoor-recovery tools such as Neural Cleanse [78] assume that a compact, class-consistent trigger can be reverse-engineered by optimizing for a target behavior. In contrast, FeatMark’s evidence is localized, identity/style-conditioned, and realized differently across samples. An attacker may infer a coarse candidate concept from the released image set, but does not observe the full defender-side program: the selected feature among semantically overlapping alternatives, the spatial grounding, the edit direction and strength, the private message-to-reader mapping, or the cloaking objective that transfers the cue through mimicry training. Consequently, concept neutralization can weaken some visible evidence but does not reliably erase the transferred semantic bias without rewriting identity/style details.

We also explicitly acknowledge the forward-security limitation. If an attacker learns the exact feature program and is willing to apply visible edits or discard many otherwise good generations, future attribution can be weakened. FeatMark is designed to raise the cost of such evasion and to expose a fidelity/identity tradeoff, not to provide perpetual cryptographic security after full feature leakage. This is why our evaluation includes targeted concept neutralization, diffusion regeneration, multi-round candidate filtering, and uncontrolled clean-image leakage.

Appendix H Ablation Study

In this section, we evaluate image cloaking perturbation strength and an uncontrolled setting where the mimicry adversary obtains non-watermarked images. Generalization across diffusion model versions is reported in Appendix G.1.

Table 13: Bit accuracy and perceptual quality of FeatMark in an uncontrolled setting with additional clean images used in mimicry attacks on VGGFace2.

\mathcal{N}_{\text{clean}}

of 0 is the default setting (*) where the attacker can only access the cloaked images to perform mimicry attacks. The number of cloaked images provided to the attacker is 4 for all the results.

$\mathcal{N}_{\text{clean}}$ WM? Bit Accuracy FID $\downarrow$ Precision $\uparrow$ Recall $\uparrow$ Coverage $\uparrow$ Density $\uparrow$ None Crop Brigh. Cont. JPEG Comb. 0* Y 0.98 0.75 0.96 0.96 0.99 0.98 180.98 0.47 0.19 0.83 0.62 N 0.03 0.04 0.04 0.04 0.04 0.05 1 Y 0.97 0.73 0.96 0.96 0.97 0.96 180.12 0.53 0.24 1.03 0.75 N 0.18 0.12 0.19 0.21 0.22 0.21 2 Y 0.98 0.68 0.95 0.96 0.98 0.97 177.95 0.53 0.32 1.06 0.79 N 0.38 0.18 0.38 0.42 0.44 0.41 3 Y 0.99 0.69 0.97 0.98 0.99 0.97 176.34 0.57 0.30 1.15 0.81 N 0.52 0.28 0.52 0.53 0.57 0.54 4 Y 0.98 0.64 0.95 0.96 0.97 0.95 174.89 0.59 0.38 1.15 0.83 N 0.58 0.29 0.56 0.58 0.63 0.61 6 Y 0.98 0.63 0.94 0.96 0.97 0.95 173.50 0.64 0.31 1.41 0.87 N 0.73 0.36 0.71 0.73 0.76 0.73 8 Y 0.99 0.68 0.97 0.98 0.99 0.99 172.10 0.59 0.39 1.26 0.87 N 0.79 0.43 0.75 0.77 0.82 0.81

Table 14: Bit accuracy and perceptual quality of FeatMark with different perturbation budget

\eta

, where

\eta=0.10

is the default setting (*).

$\eta$ WM? Bit Accuracy FID $\downarrow$ Precision $\uparrow$ Recall $\uparrow$ Coverage $\uparrow$ Density $\uparrow$ None Crop Brigh. Cont. JPEG Comb. clean N 0.03 0.04 0.04 0.04 0.04 0.05 171.81 0.65 0.28 1.33 0.83 0.01 Y 0.25 0.15 0.27 0.26 0.25 0.23 173.92 0.60 0.32 1.19 0.80 0.05 Y 0.85 0.59 0.83 0.88 0.81 0.80 178.46 0.43 0.27 0.75 0.61 0.10* Y 0.98 0.75 0.96 0.96 0.99 0.98 180.98 0.47 0.19 0.83 0.62 0.15 Y 0.99 0.81 0.99 0.99 0.99 0.98 183.57 0.43 0.18 0.80 0.59 0.20 Y 0.99 0.84 0.99 0.99 0.99 0.99 185.21 0.43 0.18 0.73 0.56

Robustness under uncontrolled settings. Our prior settings are rigorously controlled to demonstrate the performance of FeatMark. We now consider a setting where the attacker not only has access to the cloaked images, but also additional clean images from the target prior to watermarking efforts. Table 13 is the result for our metrics of bit accuracy and perceptual quality with number of clean target samples ( $\mathcal{N}_{\text{clean}}$ ) varied from 0 (default) to 8. When increasing $\mathcal{N}_{\text{clean}}$ , the bit accuracy gap is reduced, with perceptual quality remaining higher, but FeatMark continue to effectively mitigate the mimic adversary. When, $\mathcal{N}_{\text{clean}}$ = $2$ where $\mathcal{N}_{\text{clean}}$ is half of the number of cloaked images, FeatMark achieves bit accuracy with no transformation of 0.98-0.38. In the extreme case of $\mathcal{N}_{\text{clean}}$ = $6$ and $\mathcal{N}_{\text{clean}}$ = $8$ where $\mathcal{N}_{\text{clean}}$ is 1.5 or 2 times more populous than cloaked images, the accuracy gaps are too small to distinguish the watermark.

Watermarking with different perturbation budgets. We present in Table 14 an ablation study when the perturbation budget, $\eta$ in Equation 8 is varied from 0.01 to 0.20 with 0.10 as the default setting in our prior experiments. Larger $\eta$ produces larger perturbations in the cloaked images. With each $\eta$ we record the corresponding bit accuracy and perceptual quality.

From the results, a reduction in perturbation budget narrows the bit accuracy gap between watermarked and non-watermarked fake images, but improves the perceptual similarity between these images, making watermarks more difficult to be noticed. For example, when $\eta$ is 0.05, FeatMark preserves the bit accuracy with no transformation to 0.85-0.03, which is still large enough to distinguish watermarked and non-watermarked images, and achieve smaller FID of 178.46. In an extreme case where $\eta$ is 0.01, the perturbation has nearly no effect, so that bit accuracy gap drops to 0.25-0.03, which is not common and not recommended.

Appendix I Detection of Future Attacks

In this sub-section, we discuss potential threats in the future and argue how FeatMark remains effective against them.

Stronger mimicry adversaries. FeatMark’s success relies on the mimicry capabilities of the adversary, a weak adversary generating poor fake representations of the target identity is unlikely to faithfully recreate inconspicuous feature watermarks. However, this setting is not a concern, as poorly crafted fakes are indicative of imitation. On the other hand, stronger mimics will have a greater capacity to learn and replicate characteristics from the target’s source data, in which FeatMark’s visible features will also be present and transferred.

Visible watermarks raise the cost of attackers. We have recently observed, claims of impossibly strong LLM watermarks [89, 69] that are hard to remove provided adversaries are computationally bounded. Such a development may also occur in the text-to-image domain. Recent pixel based watermarks [18, 13, 12, 86, 90, 1, 52] have been bypassed by simple perturbations [37, 33]. As we have shown, these simple perturbations cannot evade our watermark detection due to the clear presence of feature-level watermark in the image. Unlike visible text watermarks [44, 95], FeatMark is naturally integrated into the image, alleviating the problem of image quality degradation. Already, FeatMark shows that watermarked features vary their location and specific forms across multiple fake images. With no clear position or form, the cost of removal is expected to increase beyond the cost of existing attacks against pixel-level watermarks.