Pixel Seal: Adversarial-only training for invisible image and video watermarking

AI summary

Overview

• Research area: Invisible digital watermarking for provenance of AI-generated and real image and video content, using deep neural embedder/extractor networks.
• Technical level: Advanced. The paper assumes familiarity with adversarial training, GAN discriminators, perceptual metrics (PSNR, SSIM, LPIPS), and U-Net architectures.
• Scope: The paper introduces Pixel Seal, a multi-bit image watermarking model trained with adversarial-only loss plus a three-stage schedule and high-resolution adaptation, and shows it can be extended to video via temporal watermark pooling.

What This Paper Is About

Invisible watermarking hides a binary message inside an image or video so that its origin can later be traced without the viewer noticing any change. Training such models is hard because robustness (surviving edits like cropping and compression) and imperceptibility (being truly invisible) pull in opposite directions, and existing methods rely on flawed perceptual proxies, need heavy tuning, and break down on high-resolution content. The paper's goal is a single training recipe that is stable, repeatable, produces invisible watermarks, and scales to both high-resolution images and video.

Key Contributions

1. Adversarial-only training. The authors remove all perceptual losses (MSE, LPIPS, etc.) and train the watermark embedder using only a cross-entropy message loss and a patch-based adversarial loss, avoiding the failure modes and manual tuning of proxy perceptual objectives.
2. A three-stage training schedule. Stage one trains only with the message loss at a large watermark scaling factor (α0 = 1.0) until bit accuracy saturates; stage two adds the adversarial loss and anneals the scaling factor to its final value (α1 = 0.2) via a cosine schedule; stage three finetunes the model at the final scaling factor for specific application needs.
3. High-resolution adaptation. The training pipeline simulates inference at full resolution: the watermark is computed at model resolution, upscaled to the original resolution, multiplied by a Just-Noticeable Difference (JND) attenuation map computed at original resolution, and the extractor input is produced by the same downsampling that will occur at inference. This removes upscaling artifacts and improves robustness.
4. Temporal watermark pooling for video. An inference-only temporal average pooling and un-pooling layer is inserted into the embedder's U-Net, allowing the image model to watermark video quickly with what the authors report as no drop in imperceptibility or robustness, without any video finetuning.

Main Findings

• Robustness gains on images. On 1000 Meta AI-generated images at 1280×1280, Pixel Seal reaches bit accuracy / −log10 p of 1.00 / 75.8 (identity), 0.97 / 68.1 (valuemetric), 0.98 / 68.9 (compression), 0.95 / 63.9 (geometric), and 0.91 / 50.3 (combined). On 100 real SA-1b validation photos it reaches 1.00 / 75.4, 0.98 / 68.5, 0.99 / 73.6, 0.93 / 59.0, and 0.94 / 55.6 respectively.
• Large margins on geometric and combined attacks. For combined attacks (cropping, JPEG compression, brightness change), the paper states that only Pixel Seal and InvisMark keep bit accuracy above 90%, while Pixel Seal embeds a much larger message (256 bits versus 94 bits for InvisMark).
• Video results. With temporal watermark pooling at step size k = 4 and depth n = 2, Pixel Seal achieves on MovieGen videos 1.00 / 76.9 (identity), 1.00 / 76.5 (valuemetric), 0.86 / 48.6 (compression), 0.97 / 68.0 (geometric), and 0.70 / 25.6 (combined); on SA-V videos it achieves 1.00 / 77.1, 1.00 / 76.8, 0.86 / 48.1, 0.98 / 70.8, and 0.68 / 24.2. Video compression (H.264, HEVC) reduces bit recovery for all methods.
• Imperceptibility. Video Seal 0.0 has substantially higher PSNR and SSIM than other methods, which the authors attribute to small but very visible localized artifacts that skew averaging-based metrics. Across the quality metrics used (PSNR, SSIM, LPIPS, CVVDP, and a JND metric), only Pixel Seal and TrustMark-P rank in the top three on 4 out of 5 metrics, and the authors note TrustMark-P is significantly less robust than Pixel Seal. The individual values in the imperceptibility table are not included in the available text.
• Qualitative watermark shape. Pixel Seal's watermarks are localized to object edges and of small magnitude, in contrast to the waves and blobs reported for other methods, and it avoids visible artifacts in single-color regions.

Methodology in Plain English

The authors start from the standard post-hoc watermarking setup: an embedder network takes an image and a binary message and outputs a small perturbation that is added to the image, and an extractor network tries to recover the message from the watermarked image after it has been altered by simulated edits. Instead of the usual weighted sum of a message loss, a perceptual loss, and an adversarial loss, they drop the perceptual term entirely and let a patch-based discriminator (the one from Rombach et al., 2022) be the only thing enforcing invisibility. A "watermark boosting" factor β scales the watermark before it reaches the discriminator, giving a knob that trades imperceptibility (β > 1) against robustness (β < 1); the released model uses β = 1. Because a pure adversarial-plus-message objective is unstable, training is split into three stages: first learn a strong but visible watermark, then gradually shrink the watermark strength while the discriminator pushes it toward invisibility, then finetune. For high resolution, they never compute the watermark at full resolution; they compute it at model resolution (256×256 in the standard pipeline), upscale it, mask it with a JND attenuation map computed on the full-resolution image, and during training send the extractor the exact downsampled input it will see at inference. For video, they observe that neighboring frames are highly correlated and average the embedder's high-level features across groups of k frames at inference only, repeating the pooled features in the decoder so per-frame low-level detail is still computed per frame.

Model details: the embedder is a U-Net-based architecture with 43.8M parameters, the extractor is ConvNext-v2 Tiny with 33.4M parameters, trained on SA-1b for 600k steps at batch size 256 with AdamW at learning rate 5×10⁻⁴, cosine decay, 20k-step linear warm-up, images resized to a random size in [S_min, S_max] with S_min = 256 and S_max = 768, message length n_bits = 256, λ_msg = 1.0 and λ_adv = 0.1, and no perceptual loss.

Why This Matters

• Research impact: The paper argues that commonly used perceptual proxies (MSE, LPIPS) and multi-loss training recipes are the root cause of visible watermark artifacts and unstable training, and shows that removing them entirely can improve both robustness and imperceptibility at once. It provides a concrete recipe (adversarial-only loss, staged scaling schedule, training-time inference simulation) that other watermarking systems can adopt, and it reports new state-of-the-art results against CIN, InvisMark, MBRS, TrustMark, Video Seal, and WAM baselines.
• Model and content provenance: Reliable multi-bit extraction under cropping and compression supports tracing which tool or uploader produced a piece of media, which the paper cites as a growing authenticity infrastructure need.
• Camera and professional devices: The JND attenuation and small α1 are motivated by applications where the watermark must be comparable to camera noise so it does not alter the photographer's intent.
• AI-generated media labeling: The evaluation uses 1000 Meta AI-generated images and Movie Gen-generated videos, directly targeting the labeling and detection of synthetic content at platform scale.
• Video platforms: The temporal watermark pooling method is presented as a practical path to watermarking video without video finetuning, addressing the inference cost that the paper says makes per-frame video watermarking impractical.

Industry relevance: All authors are affiliated with Meta FAIR, and the paper connects its work to Meta's use of watermarking for provenance and to Google's deployed SynthID, while noting that details of those deployed methods are scarce. The paper links to released code and a project website.

Future Directions

• Detailed ablations and hyperparameter studies. The paper mentions conducting ablation studies to verify its design choices, but the available text truncates before those results; isolating the effect of each component (adversarial-only loss, three-stage schedule, JND attenuation, inference simulation) is a natural next step.
• Closing the video compression gap. Pixel Seal's weakest numbers are on video compression attacks (H.264/HEVC), with combined-attack −log10 p of 25.6 on MovieGen and 24.2 on SA-V, leaving room for improvements specific to codec robustness.
• Resolution and scaling limits. Training caps image size at S_max = 768 with S_min = 256; extending the recipe to larger resolutions and to resolutions beyond what was simulated is an open question.
• Combining with generation-time watermarking. The paper explicitly positions itself as post-hoc and contrasts with in-generation methods such as Stable Signature, Gaussian Shading, and Tree-Ring; whether the two families can be combined is unaddressed.
• Generalization of the perceptual metrics. The paper relies on a JND regressor trained on internal user study data; how well that metric transfers to other populations and content types is not established in the available text.

Target Audience

Researchers and engineers working on media provenance, watermarking, or generative-model safety who already understand adversarial training and image quality metrics; practitioners at platforms and camera manufacturers who need to deploy multi-bit image and video watermarking; and graduate students studying robustness–imperceptibility tradeoffs in learned steganography and watermarking. The paper is not beginner-friendly, since it assumes fluency with GAN discriminators, U-Nets, and perceptual metric limitations.

Authors’ abstract

Invisible watermarking is essential for tracing the provenance of digital content. However, training state-of-the-art models remains notoriously difficult, with current approaches often struggling to balance robustness against true imperceptibility. This work introduces Pixel Seal, which sets a new state-of-the-art for image and video watermarking. We first identify three fundamental issues of existing methods: (i) the reliance on proxy perceptual losses such as MSE and LPIPS that fail to mimic human perception and result in visible watermark artifacts; (ii) the optimization instability caused by conflicting objectives, which necessitates exhaustive hyperparameter tuning; and (iii) reduced robustness and imperceptibility of watermarks when scaling models to high-resolution images and videos. To overcome these issues, we first propose an adversarial-only training paradigm that eliminates unreliable pixel-wise imperceptibility losses. Second, we introduce a three-stage training schedule that stabilizes convergence by decoupling robustness and imperceptibility. Third, we address the resolution gap via high-resolution adaptation, employing JND-based attenuation and training-time inference simulation to eliminate upscaling artifacts. We thoroughly evaluate the robustness and imperceptibility of Pixel Seal on different image types and across a wide range of transformations, and show clear improvements over the state-of-the-art. We finally demonstrate that the model efficiently adapts to video via temporal watermark pooling, positioning Pixel Seal as a practical and scalable solution for reliable provenance in real-world image and video settings.