HairDeck: Animation-Ready Stylized Hair Cards from a Single Image

AI summary

Overview

Research area: Computer vision and 3D graphics — single-image generation of animation-ready, stylized 3D hair assets (hair cards plus their rigs).

Technical level: Advanced. The method combines a variational autoencoder, two diffusion transformers, flow matching, and a physics-based spring-bone simulation; the paper assumes familiarity with latent generative models and character rigging.

Scope: The paper introduces HairDeck, a system that turns one image (or a point cloud) of a stylized character into a set of hair cards with per-card geometry, cross-section, orientation, and the spring-bone chains needed to animate them.

What This Paper Is About

Stylized game and avatar characters do not use realistic strands for hair; they use hair cards — a few dozen to over a hundred individually placed ribbons whose actual geometry is the visible hair — animated by short bone chains called spring bones. Artists model and rig every card by hand, and no existing image-to-3D method produces this asset: hair generators output realistic strands, image-to-3D pipelines output one fused mesh with nothing to animate, and CHARM (the only prior stylized-hair-card generator) decodes a hairstyle autoregressively as a long sequence of control points.

HairDeck's goal is to generate the whole card set at once, from a single image, together with the spring-bone chains that drive it, so the output is a directly usable, animatable asset.

Key Contributions

1. A card token for stylized hair. Each card is described by control points (position, width, thickness) plus two attributes CHARM's representation omits — the cross-section type (rhombic or triangular) and the root normal (the width direction at the root, read from the artist mesh). A card VAE compresses the variable-length control points into a single 32-dimensional latent, and the decoder reconstructs the card by arc length rather than by point index.

2. A set formulation of hair-card generation. A hairstyle is treated as an unordered set: each card is one 40-dimensional token (root position, root normal, VAE latent, cross-section type, existence flag), and a set diffusion transformer with no positional embedding denoises all 192 token slots together using flow matching. An existence flag per slot lets the model decide how many cards appear.

3. Single-image generation with a face-anchored head frame. Cards are predicted in a frame anchored to the face rather than the dataset's unit-height frame, and training renders are widened with synthetic style augmentation (halftone, XDoG line drawings, flat colors, painterly filters) plus renders restyled by an image-editing model into gacha art, thick paint, and semi-realistic painting.

4. Spring-bone chains generated as a second set. Each chain becomes a 27-dimensional token (path, joint count, stiffness, existence flag); a second set diffusion transformer generates up to 48 chains per hairstyle conditioned on the card tokens, and a fixed geometric skinning binds the cards to them for export.

Main Findings

• Large accuracy and speed gains over CHARM on point clouds. On CHARM's 100 test hairstyles, HairDeck lowers Chamfer distance from 0.0117 to 0.0066 (a 44% reduction), card-orientation error from 40.0° to 16.8° (more than half), and runtime from 33.7 s to 1.01 s (33 times faster). Voxel IoU rises from 0.753 to 0.867 and Hausdorff distance falls from 0.0498 to 0.0242.

• CHARM is roughly as accurate as retrieval. Retrieving the training hairstyle with the closest condition reaches a CD of 0.0119, close to CHARM's 0.0117, which the authors use to characterize CHARM's accuracy.

• The set formulation beats autoregression on the same tokens. A card-level autoregressive model using the same tokens reaches CD 0.0078 and orientation 16.6°; the set DiT reaches CD 0.0066 and runs 6.8 times faster. The autoregressive model also stops early, producing 60.8 cards on average against 68.9 for the set DiT and 67.4 in the ground truth.

• Strong single-image results. On 76 rendered test characters, HairDeck reaches aligned CD 0.0089 and IoU 0.725, against 0.0139/0.638 for TRELLIS.2, 0.0148/0.638 for Hunyuan3D-2.1, 0.0186/0.540 for CHARM, and 0.0174/0.592 for DiffLocks strands converted to cards. It fails on 3.5% of test characters and 10.0% of 40 in-the-wild images, versus 13.2%–34.2% and 17.5%–77.5% for the baselines, at 1.33 s per image against 15.6 s to 83.0 s.

• Cascades are worse than direct generation. Passing an image-to-3D output, sampled as a point cloud, into CHARM or into HairDeck's own point-cloud model yields aligned CDs of 0.0138 to 0.0170 and failure rates of 22.4% to 32.9% on test characters, all above HairDeck's direct 0.0089 and 3.5%.

• The VAE latent produces smoother cards than raw points. Raw control points as tokens give CD 0.0068 with a "turn" angle of 21.9°, and index-based VAE decoding gives 0.0079 with 25.3°; the arc-length VAE gives 14.2°, close to the artists' 14.1°.

• Root normal and existence flag both matter. Removing the root normal raises orientation error from 16.8° to 26.4°. Replacing the existence flag with a card-count head produces too few cards (62.8 against 68.9).

• The method is robust to data, encoder, and model size. Training on CHARM's card templates raises orientation error to 26.7°; using CHARM's encoder (Michelangelo) gives CD 0.0068, and a 54.6M-parameter model (fewer than CHARM's 61.2M) gives 0.0072 — both far below CHARM's 0.0117.

• The head frame and restyled data help the image model. Without the head frame, pre-alignment CD rises from 0.0136 to 0.0189 and reprojection IoU with the input hair drops from 0.829 to 0.798. Synthetic style augmentation lowers aligned CD on restyled inputs from 0.0100 to 0.0097, and restyled training images raise root precision from 0.781 to 0.849.

• Generated chains outperform retrieval and procedural rigs. On the 58 test hairstyles with chains, the chain DiT on artist cards reaches recall 0.625 and precision 0.585 (retrieval: 0.289/0.285; procedural: 0.397/0.110), with motion correlation 0.702 against 0.622 and 0.538, and a card fit of 0.953. On generated cards, recall is 0.542 from point clouds and 0.390 from single images, with fit staying at about 0.95.

Methodology in Plain English

Representation. Instead of predicting hundreds to thousands of discretized control-point tokens in sequence, HairDeck makes one token per card. A small autoencoder squeezes a card's variable-length polyline (position, width, thickness at each point) into a 32-number code, and that code travels with the card's root position, its root normal, a cross-section flag, and an "I exist" flag. A hairstyle is a fixed-size bag of 192 such slots, most filled, some empty.

Generating the set. A diffusion transformer looks at all slots simultaneously — its attention has no notion of position, so the cards have no order — and denoises them with flow matching, conditioned either on a point cloud (using a frozen Dora-VAE encoder) or on an image (using a frozen DINOv3 ViT-L/16 on a crop around hair and face). Empty slots let the card count emerge during denoising rather than being predicted separately. Root position and the existence flag are weighted 4× in the loss.

Putting hair on the right head. Because the dataset scales every character to unit height, heads vary in size and height, and accessories like ears can skew the scale. The authors re-map each character so its face width equals the training median (W = 0.1276) and the top center of its face sits at a common point O, then invert the mapping at test time.

Handling art styles. Every training character is rendered from 8 views in Blender with random poses, simulated hair motion, and varied cameras. Renders are then augmented with synthetic filters, and a pool of 21,336 images restyled by an image-editing model (kept only if silhouettes match, at IoU ≥ 0.85 for hair and 0.9 for the character) is sampled with probability 0.33.

Rigging. A second, smaller diffusion transformer generates the spring-bone chains as a set, conditioned on the card tokens. A fixed geometric skinning rule attaches cards to the nearest chain (within 0.4 face widths, following the lower 60% of the card's centerline, with a ramp over 0.05 face widths), tuned against the artists' own skin weights. Simulation follows the UniVRM spring-bone update at 60 fps.

Export. Every card mesh carries UVs (across the width, and arc-length fraction from root to tip), and the results can replace a VRM character's hair, hair bones, and spring-bone groups.

Why This Matters

Impact on research. The paper argues for a general rule for assets made of artist-defined primitives: one token per primitive, and no order among them. It provides direct evidence that on identical tokens, set diffusion is both more accurate and 6.8 times faster than autoregressive decoding — relevant to mesh, CAD, sketch, and scene generation, which the authors note are largely still generated as sequences.

Real-world applications.

• Game and VTuber/avatar production: turning a character illustration into riggable hair for Unity (Dynamic Bone), VRChat (PhysBones), Unreal (Kawaii Physics), or Godot.
• Rapid prototyping of character assets from concept art in many drawing styles, including gacha-style mobile game art and semi-realistic painting.
• Retrofitting animation onto existing hairstyles: the chain generator reads only card tokens, so it can rig card hair an artist modeled without spring bones.
• Design exploration through interpolation between two hairstyles by blending condition tokens, with all samples sharing initial noise.

Industry relevance. The output is deliberately a downstream-ready asset (cards, chains, UVs, skin weights, VRM export) rather than a research-only representation, and the 1.01 s to 1.33 s runtimes on a single RTX A6000 make batch or interactive use plausible. The comparison against commercial-grade image-to-3D systems (TRELLIS.2, Hunyuan3D-2.1) targets a practical gap: those systems produce a fused surface with nothing to animate.

Future Directions

• Extending beyond VRoid stylized hair. The authors state that very short hair and the alpha-textured cards of realistic hair fall outside the training data, so broadening the data domain is a natural next step.
• Texture generation. HairDeck produces card geometry and UV coordinates but no textures, leaving the appearance of the cards unaddressed.
• Filtering non-hair cards. Some VRoid characters model bows and hats with hair cards, so the system reproduces them as hair; the authors suggest filtering such cards from training data.
• Handling unusual rig layouts. The chain DiT follows layouts common among VRoid characters and generates fewer chains for atypical ones, such as many chains radiating from the crown — a coverage gap for the rigging stage.

Target Audience

Researchers and technical practitioners in 3D generative modeling and character asset pipelines — particularly those working on structured/set-based generation, single-image 3D reconstruction, and automatic rigging. It is also useful for technical artists and tools engineers in games, VTubing, and avatar platforms who need production-format hair assets, though the method sections require comfort with diffusion transformers, VAEs, and flow matching.

Authors’ abstract

The hair of stylized characters in games and avatars is built from hair cards: artist-placed ribbons whose geometry is the visible hair, animated by spring bones. Existing image-based methods produce realistic hair or a single fused mesh, while CHARM, the only generator of stylized hair cards, decodes a hairstyle autoregressively as a long sequence of control points. Observing that a card hairstyle is an unordered set of artist-defined primitives, we propose HairDeck, which generates it as one: each card is a single token, and a set diffusion transformer denoises all cards together. From a single image, HairDeck generates the cards and the spring-bone chains that drive them, yielding an animatable asset that exports to ready-to-use downstream formats. On CHARM's own test set, it lowers Chamfer distance by 44% and card-orientation error by more than half while running 33 times faster, and from single images it outperforms CHARM, image-to-3D pipelines and strand-based methods. Project page: https://hyzcluster.github.io/HairDeck