FurE: Efficient Instance-Specific 3D Fur Reconstruction without Animal-Fur Datasets

AI summary

Overview

Research area: Computer vision and 3D graphics — specifically instance-specific, strand-based reconstruction of animal fur from calibrated multi-view images.

Technical level: Advanced. The paper assumes familiarity with neural surface reconstruction, 3D Gaussian Splatting (3DGS), differentiable rendering, PCA-based hair codecs, latent strand fields, and SMAL-style body-part localization.

Scope (one sentence): The paper introduces FurE, a pipeline that reconstructs explicit, editable, per-strand animal fur from multi-view images in under one hour of strand training by learning compact latent codes on a defurred body surface, using human-hair priors instead of any animal-fur training dataset.

What This Paper Is About

Reconstructing animal fur as explicit 3D strands from photographs is hard because fur is fine, heavily self-occluded, view-dependent, and varies enormously both between species and between body parts of the same animal — and unlike human hair, there is essentially no animal-fur dataset to learn from. A further problem is that the surface recovered from multi-view images is the outer furry envelope, not the hidden skin where strand roots should sit. FurE addresses both problems: it estimates a plausible defurred body mesh from local fur-thickness cues, then optimizes compact per-root latent codes (decoded through a PCA basis borrowed from human hair) rather than dense per-strand geometry, cutting strand training to under one hour.

Key Contributions

1. FurE, an instance-specific fur reconstruction method producing explicit, editable animal fur from calibrated multi-view images, with strand training completed in under one hour — a reported 10× speedup over state-of-the-art methods at comparable or better rendering quality.

2. Transfer of human-hair priors to animal fur through a compact PCA codec. Dense strand optimization is replaced by root-conditioned latent learning, and no animal-fur training data is used.

3. Shell-based defurring that combines local Gaussian Frosting cues with part-aware priors to estimate a plausible hidden strand-root surface, without SMAL fitting.

4. The first instance-specific, strand-based animal fur reconstruction from noisy real-world multi-view images, demonstrated on a bison sequence where NeuralFur fails (to the authors' knowledge).

Main Findings

• Strand training time: NeuralFur takes 10.5 h of strand training versus 52 min for FurE; preprocessing takes 10 h versus 1 h. Both compared using 36 views on an A5000 GPU (Table 1). The paper summarizes this as a 10× speedup and strand training "in under one hour."

• Strand accuracy on a synthetic tiger with artist-generated ground-truth strands (Table 2, thresholds given in cm/degrees): FurE reaches a precision of 27.62 / 41.78 / 51.20, a recall of 21.69 / 38.00 / 50.52, and an F-score of 24.30 / 39.80 / 50.86 at the 2/20, 3/30, and 4/40 thresholds. NeuralFur scores 26.22 / 39.32 / 48.05 precision, 20.58 / 34.08 / 45.69 recall, and 23.06 / 36.51 / 46.84 F-score. GaussianHairCut scores 16.24 / 25.51 / 32.34 precision, 23.51 / 36.04 / 45.87 recall, and 19.21 / 29.87 / 37.93 F-score. FurE is highest on all three F-scores.

• Unsupervised geometry consistency across four scenes (Panda, Fox, Cat, whiteTiger), Table 3: FurE is close to NeuralFur and clearly better than GaussianHairCut on length statistics. For Panda, the global mean strand length μ_L is 5.22 for FurE, 5.23 for NeuralFur, and 8.79 for GaussianHairCut, with length standard deviation σ_L of 1.38, 1.38, and 2.04 respectively. For Fox, μ_L is 3.80 (FurE), 3.00 (NeuralFur), and 7.190 (GaussianHairCut), with σ_L of 1.15, 1.18, and 3.18. For Cat, μ_L is 1.60, 1.65, and 4.94; for whiteTiger, 3.70, 4.00, and 6.20. Local direction variance on Panda is 0.046 (FurE), 0.043 (NeuralFur), and 0.56 (GaussianHairCut). The paper notes there are no ground-truth strands for these metrics.

• Rendering quality across four scenes (PSNR ↑ / LPIPS ↓ / SSIM ↑), Table 4: Panda — FurE 43.91 / 0.3258 / 0.6144 versus NeuralFur 43.91 / 0.3266 / 0.6135 and GaussianHairCut 43.82 / 0.3281 / 0.6130. whiteTiger — 43.10 / 0.3097 / 0.6187 versus 43.11 / 0.3107 / 0.6181 and 43.09 / 0.3128 / 0.6169. Fox — 44.88 / 0.3261 / 0.6092 versus 44.87 / 0.3268 / 0.6090 and 44.86 / 0.3284 / 0.6082. Cat — 49.84 / 0.3120 / 0.6165 versus 49.82 / 0.3125 / 0.6132 and 49.82 / 0.3141 / 0.6141. The paper describes FurE as "largely outperforming" the baselines on rendering while being roughly 10× more efficient in training; the numerical margins themselves are small.

• Real-world generalization: On a noisy real-world bison multi-view sequence, FurE reconstructs strand-based geometry while NeuralFur fails to capture the underlying geometric surface, which the paper says prevents NeuralFur from recovering coherent fur. The associated multi-view data is released.

• Ablations: Using a fixed strand length uniformly across all body parts leads to inaccurate fur reconstruction, particularly on the body and belly where strands are naturally longer. Removing the defurred mesh yields visually similar results but produces quantitative inconsistency in strand direction and curvature.

• Physics and pipeline compatibility: Reconstructed strands imported into Blender and Unreal Engine respond with realistic, physically coherent motion under simulated strong wind (Figure 9 / Figure 12).

Methodology in Plain English

The pipeline has two stages and is optimized end to end from images inside a 3DGS-compatible framework.

Stage 1 — Figuring out where the skin is (defurring). The method first reconstructs the visible furry outer surface with NeuS2 from calibrated images and foreground masks. It then runs Gaussian Frosting — a representation that surrounds a base mesh with an adaptive layer of 3D Gaussians — to get local shell widths, which act as cues about where fur is thick. These widths are not treated as direct measurements of skin depth or strand length. Body parts are labeled with ALIGN-Parts (with manual verification) rather than fitting a SMAL template. Each outer-mesh vertex is then pushed inward along its normal by an estimated displacement, solved as a smoothness-regularized bounded optimization: local Frosting evidence pulls the displacement where correspondence support is strong, part-level thickness references guide it where support is weak, smoothing is applied across mesh edges (weaker across part boundaries), non-fur regions such as eyes, nose tip, paw pads, and horns stay fixed, and displacements that would cause face flips, collapse, or new self-intersections are reduced.

Stage 2 — Learning strands as compact codes instead of dense geometry. Strand lengths are initialized separately from root displacement, because a curved or oblique strand can be longer than the coat is thick. For each fur-bearing part, a length statistic is taken as the 75th percentile of that part's shell widths converted to centimeters, multiplied by a fixed part-specific multiplier (variant V1). An optional VLM-assisted variant (V2) blends 60% of this shape-based length with 40% of a VLM length estimate, clipped to between 35% and 135% of the VLM value. Roots are sampled on the defurred mesh at part-specific densities, with zero or near-zero density in non-fur regions. For every root, a small encoder predicts a low-dimensional coefficient vector from the root's positional encoding, part label, normalized local Frosting thickness, and target length. A PCA decoder — initialized from PERM's human-hair basis — turns that vector into a normalized local strand curve, which is anchored at the root, rotated into the root's tangent–bitangent–normal frame, and scaled to the target length. The codec parameters are optimized and the decoder fine-tuned per instance using the target animal's own images, with no animal-fur training set.

Rendering and supervision. Each strand segment becomes an anisotropic cylindrical Gaussian whose length matches the segment and whose orientation follows the local tangent, with trainable spherical harmonic coefficients for appearance. These Gaussians are rendered differentiably and supervised by a weighted combination of losses: RGB, rendered-mask silhouette, image-space orientation, a "charm" loss pulling fur toward the outer envelope, an SDF loss preventing penetration into the body, and a fur-mask loss.

Training budget reported: 15K strands per iteration for 2500 iterations, with the same hyper-parameters as NeuralFur. Defurring optimization uses L-BFGS-B for at most 2,500 iterations (ftol=1e-13, gtol=1e-7, maxcor=8), with weights η=0.25 and λ=8. The strand width parameter is initialized as clip(0.03·ℓ_p, 0.02, 0.12), and image-space fur direction cues come from a bank of 180 Gabor orientations. Metric scale is derived from the ratio of an assumed eye separation in centimeters to the reconstructed eye separation in scene units.

Data used: Five synthetic fur styles across different animals from the Artemis dataset, trained on 36 uniformly sampled frames per sequence and evaluated on the remaining frames for novel-view rendering, plus a real-world bison sequence. Baselines include GaussianHairCut, NeuralFur, GenZoo, and Neural Haircut.

Why This Matters

Impact on research. Fur capture has lagged human hair capture, largely because human hair has large-scale strand datasets and animal fur does not. FurE shows that human-hair priors can be transferred to animal fur through a compact PCA codec, and that the expensive part of the problem can be moved from dense 3D strand space into a low-dimensional latent space. It also shows that defurring — recovering the hidden skin surface — can be framed as a local shell-estimation problem using Gaussian Frosting cues plus part priors, removing the dependence on SMAL template fitting. That combination is what makes the reported 10× training-speedup possible while keeping output as explicit, editable polylines rather than an implicit or volumetric representation.

Real-world applications:

• Digital asset creation for film, animation, and games, where explicit strands are the production-standard representation because they remain directly editable, renderable, and simulatable.
• Real-time pipelines in Blender and Unreal Engine, demonstrated in the paper, including physics simulation such as wind dynamics on the reconstructed strands.
• Animal pose estimation, 3D part segmentation, and tracking, which the paper notes depend on disentangling fur from the underlying body.
• Digitization of rare or hard-to-scan animals from ordinary multi-view capture, since the method works from noisy real-world images (demonstrated on a bison) and avoids animal-fur training data.

Industry relevance. The output format is the one studios already use. Converting captured fur into explicit strands instead of a volumetric proxy means the result can be groomed, edited, simulated, and rendered without a retopology or resampling step. Cutting strand training from 10.5 h to 52 min and preprocessing from 10 h to 1 h changes the economics of per-shot capture, and the method's tolerance of noisy real-world inputs matters more than performance on clean synthetic assets.

Future Directions

• Ground truth with strong within-part variation. The authors state their evaluation lacks ground-truth cases such as shaved patches, injuries, or irregular grooming, which would better test local Frosting cues against part-level priors.

• Multi-layer coats and joint root/strand optimization. FurE currently represents fur as a single strand layer on one root surface estimated before strand optimization. Modeling multilayer coats and optimizing root placement jointly with strand geometry under multi-view supervision are named as important next steps.

• Dynamic capture. The framework requires calibrated multi-view images of a static animal, and the authors note that capturing dense static multi-view data of living animals is very difficult; extending to dynamic motion is the next step.

• Robustness to occlusion and missing metric scale. Severe self-occlusion from the animal and its fur degrades shell-thickness and fur-length estimates, motivating stronger geometric priors. Separately, centimeter-valued length initialization requires a metric reference, and the current procedure has no validated automatic fallback when metric information is unavailable.

Target Audience

Researchers and practitioners in 3D reconstruction, neural rendering, and computer graphics who work on hair or fur capture, neural avatars, or Gaussian-Splatting-based scene reconstruction. It is also relevant to technical directors and pipeline engineers in VFX, animation, and games who need editable groom assets produced quickly from multi-view capture, and to computer-vision researchers working on animal body reconstruction, part segmentation, and pose estimation who need the fur layer disentangled from the body. The paper is not beginner-friendly: it assumes comfort with SDF-based reconstruction, 3D Gaussian Splatting, differentiable rendering, and latent/PCA geometry representations.

Authors’ abstract

Realistic and editable animal fur reconstruction from multi-view images is challenging due to fine-scale detail, self-occlusion and obfuscation, and, unlike human hair, the lack of animal-fur datasets. Fur usually covers most of an animal's body, with large inter-species and intra-species variability. We present FurE, an efficient strand-based animal fur reconstruction method that recovers a per-strand, editable groom by optimizing a root-conditioned latent field, decoded into strand geometry via a PCA-based decoder. We reconstruct a defurred animal body using local fur-thickness cues from a surface-constrained Gaussian Frosting representation together with part-based priors. We further show that a PCA-based decoder learned from human-hair strand data can alleviate animal-data scarcity while enabling substantially faster optimization. FurE achieves a 10x speedup in strand training over current SOTA dense per-strand optimization while retaining strand fidelity and generalizing across synthetic and real-world sequences, with quantitative and qualitative validation despite the reduction in training time.