Tune-Your-Style: Intensity-tunable 3D Style Transfer with Gaussian Splatting

AI summary

Overview

• Research area: 3D style transfer for 3D Gaussian Splatting (3DGS) scenes, combining diffusion-model priors with explicit 3D scene representations.
• Technical level: Advanced. The paper assumes familiarity with 3D Gaussian Splatting, diffusion models, cross-attention/self-attention feature injection, and standard stylization losses (L1, LPIPS).
• Scope: The paper introduces Tune-Your-Style, a paradigm in which a user-adjustable "style tuner" lets a rendered 3D scene receive any desired amount of style, from no style to full style, instead of producing a single fixed stylization.

What This Paper Is About

3D style transfer must balance the original content of a scene against the patterns and colors of a reference style image, and existing methods pick one fixed balance point for all users. The authors argue this fixed-output paradigm cannot accommodate different users' preferences, so they build a system where the style intensity injected into the scene is a continuous, user-controlled parameter. The goal is a 3DGS-based pipeline that produces high-quality stylized scenes while letting users dial style intensity up or down after training.

Key Contributions

1. A new intensity-tunable 3D style transfer paradigm. The paper frames the fixed content-style balance of prior work as a limitation and replaces it with a user-adjustable style intensity.
2. Explicit style intensity modeling plus a learnable style tuner. Gaussian neurons predict attribute offsets for all Gaussian attributes (not just color), and a learnable tuner parameterizes how much of those offsets is applied.
3. Tunable Stylization Guidance (TSG). A diffusion-based guidance scheme that produces multi-view consistent stylized views via cross-view style alignment, then trains with a two-stage optimization that interpolates between zero-style guidance from the initial rendering and full-style guidance from the stylized views.
4. Extensive comparison and application demonstrations. Qualitative and quantitative comparisons against 3DGS-based and NeRF-based methods, a user study, and visualizations of two applications: intensity-tunable stylization and multi-style combination.

Main Findings

• Best multi-view consistency in the quantitative comparison. On the "LPIPS / RMSE" format (lower is better), the method scores 0.033 / 0.035 for short-range consistency and 0.062 / 0.067 for long-range consistency, versus StyleGaussian at 0.067 / 0.070 and 0.126 / 0.108, G-Style at 0.044 / 0.059 and 0.093 / 0.096, and InstantStyleGaussian at 0.053 / 0.062 and 0.108 / 0.113.
• Strongest style fidelity by CLIP metrics. The method reaches CLIP similarity 0.2619 and CLIP directional similarity 0.2881, against G-Style (0.2406 / 0.2391), InstantStyleGaussian (0.2204 / 0.2160), and StyleGaussian (0.2134 / 0.2223).
• Highest user preference. In a user study with 20 participants scoring content consistency, style consistency, and visual appeal on a 5-point scale, the method scores 3.97 ± 0.13, compared with G-Style at 3.10 ± 0.40, StyleGaussian at 2.79 ± 0.16, and InstantStyleGaussian at 2.06 ± 0.22.
• Clearer textures and fewer artifacts qualitatively. Against 3DGS-based methods on five scenes (face, truck, train, lego, garden), the authors report that prior methods show blurring and artifact noise, while their method preserves content, keeps color fidelity with the reference style, and produces fewer artifacts. Additional NeRF-based comparisons on LLFF scenes (fern, horns, orchids, trex, flower) appear in the appendix.
• Two-stage optimization is necessary for tunability. Removing it causes over-stylization and compromises the style tuner's ability to adjust intensity, which the authors attribute to unstable supervision from randomly mixing zero-style and full-style guidance.
• Cross-view style alignment preserves content at large viewpoint changes. Without alignment, stylized views lack 3D consistency; with feature injection alone, views far from the anchor view become distorted; adding content calibration keeps the current view's content and layout while matching the anchor's style texture.
• Two applications are demonstrated. Intensity-tunable 3D stylization shows rendered results across a range of tuner values, and multi-style combination injects two styles into masked interior and exterior regions with independently adjustable intensities.

Methodology in Plain English

The pipeline starts from a scene already reconstructed as 3D Gaussians using the standard 3DGS method.

Modeling style as offsets. For each Gaussian, a small learnable "neuron" predicts offsets for every attribute: position, scaling, rotation, opacity, and color. Because previous work typically modifies only color, predicting all attributes lets the system stylize both geometry and appearance. A reference style image conditions these predictions.

Making style intensity adjustable. A "style tuner" has two endpoints, 0% (no style injection) and 100% (full style injection). A staircase function quantizes the continuous tuner value into discrete levels, which map to learnable embeddings, and the offsets are multiplied element-wise by the selected embedding. The quantization level is set to 10 and the endpoints to 0% and 100%. Users can therefore select a level that matches their preferred content-style balance.

Filtering redundant Gaussians. Some Gaussians are barely visible in the original rendering but cause artifacts once visibility and occlusion relationships change after stylization. The authors compute an importance score per Gaussian across training views and filter out the least important 50%.

Generating stylization targets. Instead of time-consuming VGG-feature optimization, the method renders views and stylizes them with a 2D diffusion model, IP-Adapter-SDXL, injecting the reference style image feature into the first up_block. These stylized views, however, are not 3D-consistent across viewpoints.

Cross-view style alignment. One training view is chosen as an anchor. The anchor's key and value features are warped to each target viewpoint (by back-projecting with the anchor's predicted depth map and re-projecting into the target camera), then concatenated with the target's own features inside the mutual self-attention layers of the U-Net up_blocks. The warping step is the "content calibration" that prevents distortion when viewpoint differences are large.

Two-stage training. In stage one (full-style guidance), neurons and the full-offset tuner embedding are optimized against the stylized views using L1 and LPIPS losses; neurons are initialized to zero and tuner embeddings to one so early predictions are zero offsets. In stage two, those parameters are frozen and the remaining tuner embeddings are learned; at each iteration a tuner value is sampled and the loss is a weighted sum of zero-style guidance (matching the original rendering) and full-style guidance, with the sampled tuner value as the weight. Each stage runs 2000 steps for a total of 4000.

Setup. Training uses PyTorch on a single 32GB NVIDIA V100 GPU and takes approximately 20 minutes per scene. Original Gaussians come from Mip-NeRF, Tanks&Temples, LLFF, and Instruct-N2N data. Multi-view consistency is measured with LPIPS and RMSE on warped view pairs (10 view pairs per scene, intervals of 2 for short-range and 10 for long-range, using optical flow based on softmax splatting), and style fidelity uses CLIP ViT-L/14 for similarity and directional similarity.

Why This Matters

The paper reframes 3D style transfer from "produce one stylized scene" to "produce a controllable stylization family," which changes what users can do with a stylized asset after training. Because the scene is represented with 3D Gaussians, the pipeline keeps the fast training and rendering that make 3DGS attractive, and the reported ~20 minutes per scene on a single V100 keeps the approach within reach of typical research and production hardware.

Real-world applications the paper points to or demonstrates:

• Artistic creation and digital art, where an artist needs to tune how strongly a style dominates a scene rather than accept a single result.
• Gaming and entertainment, where stylized 3D assets may need different looks for different worlds or moods.
• Content customization, where different users of the same scene want different content-style balances, which the style tuner supports directly.
• Multi-style scene editing, where segmented regions of a scene receive different styles with independently adjustable dominance, as demonstrated using SAM-based masks.

Industry relevance lies in the combination of an efficient 3DGS pipeline with diffusion priors and an explicit, learnable control knob. A tunable parameter is a practical feature for interactive tools in a way that retraining per preference is not.

Future Directions

• Overcoming dependence on the 2D diffusion model. The limitations section states that the style transfer capability is primarily limited by the underlying 2D diffusion model, with the visible truncation noting image style transfer has a higher success rate than 3D. Improving stylization success for hard 3D cases is an open problem.
• Extending control beyond a single global intensity. The demonstrated multi-style combination, using SAM-derived 3D masks and per-style intensities, suggests finer spatial and per-style control as a direction.
• Broadening evaluation. The quantitative comparison covers five scenes and a user study with 20 participants; larger-scale and more diverse evaluations would test whether the reported consistency and preference advantages generalize.
• Reducing training cost. At 4000 total optimization steps and roughly 20 minutes per scene on one 32GB V100, faster or more memory-efficient tunable stylization would widen practical adoption.

Target Audience

Researchers working on 3D style transfer, 3D Gaussian Splatting, and diffusion-guided 3D editing will get the most from this paper, particularly those interested in controllable or customizable generative pipelines. Graphics, technical art, and interactive media engineers evaluating stylization for production assets also benefit, though they should expect advanced material on Gaussian attribute parameterization and diffusion attention mechanisms.

Authors’ abstract

3D style transfer refers to the artistic stylization of 3D assets based on reference style images. Recently, 3DGS-based stylization methods have drawn considerable attention, primarily due to their markedly enhanced training and rendering speeds. However, a vital challenge for 3D style transfer is to strike a balance between the content and the patterns and colors of the style. Although the existing methods strive to achieve relatively balanced outcomes, the fixed-output paradigm struggles to adapt to the diverse content-style balance requirements from different users. In this work, we introduce a creative intensity-tunable 3D style transfer paradigm, dubbed \textbf{Tune-Your-Style}, which allows users to flexibly adjust the style intensity injected into the scene to match their desired content-style balance, thus enhancing the customizability of 3D style transfer. To achieve this goal, we first introduce Gaussian neurons to explicitly model the style intensity and parameterize a learnable style tuner to achieve intensity-tunable style injection. To facilitate the learning of tunable stylization, we further propose the tunable stylization guidance, which obtains multi-view consistent stylized views from diffusion models through cross-view style alignment, and then employs a two-stage optimization strategy to provide stable and efficient guidance by modulating the balance between full-style guidance from the stylized views and zero-style guidance from the initial rendering. Extensive experiments demonstrate that our method not only delivers visually appealing results, but also exhibits flexible customizability for 3D style transfer. Project page is available at https://zhao-yian.github.io/TuneStyle.