Residual Transferability in Neural Image Watermarking

AI summary

Overview

Research area: AI security and content provenance — specifically, the security of neural (deep learning based) invisible image watermarking against forgery attacks.

Technical level: Intermediate. The paper is readable without deep cryptographic background, but it assumes familiarity with encoder–decoder watermarking pipelines, contrastive learning, and common image distortions.

Scope: The paper asks why watermark residuals can be lifted from one image and pasted onto an unrelated image, defines a metric to measure that property, isolates the architectural design choices that cause it, and proposes a plug-in defense called CoverLock. (Note: the supplied content is truncated, so the appendices referenced throughout — Appendix B, D, E, G and Table 5 — are only partially available.)

What This Paper Is About

Neural watermarks can be forged by averaging a few released watermarked images, estimating the watermark "residual," and adding that residual to a completely unrelated image so it gets falsely attributed to the original source. Prior work showed these attacks work, but nobody had explained why a residual robbed from one image still decodes on another. This paper formalizes that property as residual transferability (RT), shows it is governed mainly by architecture rather than training recipe, and introduces a defense, CoverLock, that makes verification depend on the image itself.

Key Contributions

1. A method-independent metric for residual transferability. The authors define RT@s = max(2·ā_s − 1, 0), computed from ground-truth (GT) residuals rather than from any particular attack's estimation procedure, so the metric characterizes the watermarking system itself rather than the attacker. It is evaluated with s = 100 source images, T = 100 target images, and M = 5 messages, and normalized to [0, 1] where 0 means random guessing.

2. Evidence that architecture, not training, drives RT. Comparative analysis across five systems (CIN, VINE, MBRS, HiDDeN, RivaGAN) and controlled interventions lead the authors to conclude that common training-side variations do not account for the large RT gaps, and that architectural design plays a central role.

3. Two architectural mechanisms that suppress RT. Through interventions on low-RT systems, the authors identify (a) a Broadcast–GAP co-design, in which the message is spatially broadcast and fused with local image features while the decoder keeps spatial representations until a late global average pooling stage (found in HiDDeN), and (b) content-adaptive attention that modulates the embedding pattern based on the input image (found in RivaGAN).

4. CoverLock, a plug-and-play image-binding wrapper. CoverLock XORs the message with an image-conditioned binary code derived from frozen DINOv2 features before embedding, and recomputes that code at detection. It requires no architectural change or retraining of the underlying watermarking model and is trained without access to the watermark encoder, decoder, or any forgery samples.

Main Findings

• RT varies enormously across systems. In Table 1, RT@100 ranges from 1.00 (CIN, an INN architecture trained on COCO at 128×128 with 30 message bits) and 0.99 (VINE, CNN, OpenImages, 512, 100 bits) and 0.98 (MBRS, CNN, ImageNet, 128, 256 bits) down to 0.28 (HiDDeN, CNN, COCO, 128, 30 bits) and 0.08 (RivaGAN, CNN, Hollywood2, 256, 32 bits). The authors note that even methods trained with similar objectives and robustness-oriented augmentations exhibit substantially different transferability.

• High-RT systems look cover-agnostic from two angles. Averaging residuals over groups of five cover images reveals substantially more consistent residual patterns across image groups for high-RT methods, whereas low-RT residuals vary more strongly with the cover. At the decoder's final feature-extraction layer, normalizing cover-induced and message-induced feature variances to sum to 100%, high-RT decoders show a substantially smaller relative contribution from cover variation, while low-RT systems retain a much larger cover-induced component.

• Broadcast–GAP is causal for low RT. Removing the BG design from HiDDeN sharply increases RT, and introducing it into high-RT watermarks substantially reduces RT. The authors note that they also tried to introduce BG directly into VINE, but the modified model did not converge reliably given VINE's already challenging optimization. Numerical results for these interventions are in the paper's Appendix Table 5, which is not in the supplied content.

• Content adaptivity is a second causal mechanism. RivaGAN's low RT is attributed to content-adaptive attention in its encoder that modulates the embedding pattern according to the input image; replacing it with a fixed embedding pattern causes a large increase in RT.

• Training dynamics corroborate the shortcut explanation. High-RT configurations reach high training decoding accuracy after relatively few images, consistent with an easier cover-independent shortcut, whereas configurations containing the low-RT mechanisms require substantially more image observations before converging.

• CoverLock drives forgery success to near zero. Across three high-RT systems (CIN, MBRS, VINE), three datasets (COCO, ImageNet, DIV2K) and three attacks (GT@1, Yang et al. 2024a with 100 references, and WmForger/Souček et al. 2026 with a single reference), the highest average ASR for CoverLock remains below 0.25%. The averaged ASR figures are 0.243 (GT@1), 0.015 (Yang et al. 2024a) and 0.059 (WmForger), versus 4.747 / 26.800 / 1.560 for MHDW and 12.226 / 14.216 / 65.379 for the ConvNeXt classifier baseline.

• Classifier-based defenses generalize poorly across attacks. The residual artifacts a ConvNeXt classifier learns from GT-based forgeries change substantially under different estimation procedures, so the baseline is attack-dependent and the authors warn this may lead to an iterative attacker–defender cycle. MHDW similarly becomes less effective when the attacker obtains cleaner residuals through stronger averaging or more accurate estimation.

• CoverLock's robustness gain comes from the image-dependent component. In the BER decomposition, MHDW and CoverLock-G share the same payload BER of 7.4%, but CoverLock-G reduces the hash BER from 14.6% to 4.2%, producing a substantially lower final BER.

• CoverLock keeps more accuracy under distortions than MHDW. Averaged over six distortions (Gaussian noise σ = 0.05, brightness ×1.2, Gaussian blur σ = 1.0, JPEG Q = 70, contrast ×1.2, center crop to 80% area), average BitAcc is 94.60 vs 84.09 (MHDW) for CIN, 89.40 vs 81.39 for MBRS, and 87.75 vs 83.12 for VINE. Vanilla, undefended averages are 99.36, 92.55 and 91.61 respectively. All three methods lose heavily on center crop for VINE (about 50%).

• A better security–robustness frontier. For MBRS on COCO, reducing the classifier baseline's ASR costs substantial TPR, while high-TPR operating points remain vulnerable; MHDW is similarly less favorable. CoverLock variants stay in the low-ASR, high-TPR region, and scaling the DINOv2 backbone from Small to Giant further reduces ASR while maintaining TPR at approximately 92%, placing CoverLock-G on the favorable Pareto frontier.

Methodology in Plain English

The authors start by removing the attacker from the equation. Instead of measuring how well a specific forgery attack works, they compute the ground-truth residual — the exact pixel difference between a clean image and its watermarked version — average it over 100 source images for each of 5 messages, add it to 100 unrelated target images, and see how many bits the decoder gets right. That gives RT@100, a clean read on how "portable" a system's watermark evidence is.

Then they look at behaviour. They average residuals over small groups of images to see which structures survive across covers, and they measure, at the decoder's final layer, how much the learned features change when you swap the cover versus when you swap the message. High-RT systems turn out to be barely affected by the cover; low-RT systems are strongly affected.

To test causality rather than correlation, they perform surgical edits: strip the Broadcast–GAP design out of HiDDeN, graft it into high-RT systems, and swap RivaGAN's content-adaptive attention for a fixed embedding pattern. They also plot training curves to see whether the shortcut is learnable faster.

For the defense, CoverLock wraps the message: it takes a frozen DINOv2 backbone, extracts 16×16 = 256 patch features at 224×224 input, summarizes them by channel-wise mean and standard deviation, projects them through a small trainable head (hidden dimension 512, output embedding D = 512) with ℓ2 normalization, and multiplies by a fixed, QR-orthogonalized projection matrix (native code length L0 = 256, sub-selected to the payload length L). Thresholding the resulting logits at zero yields the binary code that is XORed with the user message before embedding and XORed back after decoding. Training uses only MSCOCO natural images and their transformed views, with a symmetric contrastive loss plus stability, bit-balance, bit-variance (ν = 0.5) and binary-confidence regularizers — no watermark model and no forgery samples involved.

Why This Matters

Impact on research. The paper reframes residual-based forgery from an attack-engineering problem into a property of the watermarking architecture itself. That shift matters because building ever-stronger attacks and ever-broader classifiers is an arms race; identifying cover dependence as the underlying design principle gives the field something structural to optimize for. The RT metric also gives a common yardstick that does not need to be recomputed every time a new attack appears.

Real-world applications:

• AI content provenance and attribution. Google (SynthID), Meta, OpenAI and Amazon have integrated watermarking into generative AI ecosystems; systems with high RT could be abused for false ownership or provenance attribution with only one or a few released watermarked images.
• Platform-scale moderation and takedown disputes. Platforms deciding whether an image "belongs to" a creator need verification that survives adversarial forgery, not just benign edits.
• Model and dataset licensing. Attribution of generated outputs to a licensed model or dataset depends on watermarks that cannot be transplanted onto unrelated content.
• Retrofitting deployed watermarkers. Because CoverLock is a plug-in wrapper trained independently and requiring no architecture changes or parameter updates to the host watermarker, it can be applied to systems already released.

Industry relevance. The threat model is deliberately practical: the adversary sees one or a few released watermarked images, does not know the deployed method or any protection mechanism, has no access to encoder or decoder parameters, and cannot query the encoder. That is exactly the position a real attacker is in. The defense's selling point — no redesign, no retraining of the core model — matches how commercial deployments actually get patched.

Future Directions

• Robustness under stronger or unseen transformations and distribution shift. The authors state in their limitation section that the security–robustness balance may change under more severe or unseen image transformations.
• Adaptive attacks against the image-binding mechanism itself. The paper explicitly notes that CoverLock's behavior under adaptive attacks specifically targeting image binding remains to be studied.
• Overhead reduction. CoverLock adds an additional image-binding module plus associated training and deployment cost; reducing that overhead is an open engineering direction.
• Alternative routes to cover dependence. The authors acknowledge that other architectures may achieve low RT through design principles different from the two identified here, leaving the general space of cover-binding designs unexplored.

Target Audience

This paper is most useful to watermarking researchers and practitioners building or auditing post-processing neural watermarking systems — the encoder–decoder, model-agnostic methods the paper focuses on, as distinct from in-generation watermarking. It also serves security researchers studying adversarial forgery of provenance signals, and engineers or policy-adjacent readers at organizations deploying watermarking who need to know whether a released system is forgeable and what retrofit options exist. Readers with a basic grasp of neural network training loops, losses, and image distortions will get the most out of it; the RT metric itself is simple enough to be applied without deep theoretical background.

Authors’ abstract

Neural image watermarks can be forged by extracting watermark-bearing residuals from released images and transferring them to unrelated content. While prior work has demonstrated this vulnerability, what makes these residuals transferable remains poorly understood. We formalize this vulnerability with \textbf{residual transferability (RT)}, a metric that quantifies how well watermark evidence remains decodable after transfer across unrelated images. Through comparative analyses and controlled interventions, we find that common training-side variations do not account for the large RT differences across watermarking systems; instead, architectural design plays a central role. By contrasting high- and low-RT systems and validating their architectural differences through controlled interventions, we identify two mechanisms that strengthen the dependence of watermark evidence on the cover image, thereby suppressing the residual transferability. These findings provide concrete design guidance for developing more forgery-resistant watermarking architectures. Complementarily, for existing watermarking systems where architectural redesign is impractical, we introduce \textbf{CoverLock}, a plug-and-play strategy for existing watermarking systems that strengthens such image dependence without architectural redesign. Across representative watermarking systems exhibiting high residual transferability, CoverLock achieves a more favorable security--robustness trade-off than both traditional handcrafted defenses and learned classifier-based defenses.