TailorRig: Controllable and Structured Auxiliary Skeleton Generation for Garment-Aware Rigging

AI summary

Overview

Research area: Computer Vision / 3D character animation — automatic humanoid rigging, specifically skeleton generation and skinning for characters with complex garments.

Technical level: Advanced. The paper assumes familiarity with kinematic skeleton trees, linear blend skinning (LBS), autoregressive sequence models, cross-attention architectures, and standard rigging metrics.

Scope: TailorRig is a two-stage framework that reformulates garment-aware humanoid rigging as a controllable, structured full-rig generation problem, letting artists specify the body skeleton template and the auxiliary garment-bone categories, then generating the rig and skinning weights under a hierarchical chain-to-bone structure.

What This Paper Is About

Modern animated characters wear skirts, capes, coats, braids and layered accessories that need their own "auxiliary" bones to deform believably, but existing automatic rigging methods infer bones from geometry alone and give artists little say over which garment bones appear or where. Dense sets of garment bones packed close together also cause "weight bleeding," where a skirt erroneously follows arm motion. TailorRig reframes the task from automatic rig recovery to artist-driven rig authoring, in which the body skeleton template and the requested auxiliary categories are explicit inputs to the system.

Key Contributions

1. A controllable rigging formulation that exposes two explicit artist control axes — a user-specified body skeleton template (preserving compatibility with existing animation libraries) and semantic labels naming the auxiliary categories to generate (e.g., "skirt").

2. A structured full-rig generation framework that injects a shared 21-class semantic taxonomy and kinematic-chain hierarchy into body alignment, auxiliary-bone generation, and skinning, using one hierarchical skeleton encoder architecture with stage-specific parameters.

3. Hierarchical chain-to-bone skinning, which first assigns each vertex to a kinematic chain and then refines per-bone weights within that chain, suppressing weight bleeding between spatially adjacent but semantically unrelated structures.

4. Human-Mix, a dataset of 38,088 rigged humanoid characters with 21-class semantic bone annotations, of which 7,824 contain at least one auxiliary bone category.

Main Findings

• Best skeleton accuracy on every metric. Body alignment: J2J 0.023, J2B 0.013, B2B 0.011, versus the best template-based baseline TARig at 0.061/0.045/0.034 and template-free UniRig at 0.040/0.031/0.020. The paper reports a 42.5% reduction of UniRig's 0.040 J2J, noting UniRig is template-free and therefore a reference rather than a same-input baseline.

• Larger gains on auxiliary bones. Auxiliary J2J drops from UniRig's 0.191 to 0.068 (with J2B 0.096 and B2B 0.098), which the authors attribute to label-conditioned generation better localizing garment-aware chains. Baselines such as MagicArticulate (0.202/0.209/0.197) and Puppeteer (0.201/0.211/0.198) are far behind.

• Best skinning in the controlled setting. Under identical ground-truth skeletons, TailorRig reaches scaled L1 7.70 and Def-L1 0.60, compared with RigNet 8.56/2.14, Puppeteer 10.31/0.64, UniRig 11.55/0.91, MIA 14.43/1.56, and NBS 62.86/28.52. This isolates weight prediction from skeleton error.

• Labels genuinely govern generation. Requesting one ground-truth auxiliary category per test character, TailorRig achieves 92.6% requested-bone recall with only 7.9% off-label bones, versus UniRig at 42.1% / 25.1%. The authors note the capsule-inheritance mapping used to score UniRig is favorable to it.

• Both semantic and geometric structure matter. Removing the semantic classification loss degrades auxiliary J2J from 0.068 to 0.120 and skinning L1 from 7.70 to 11.50; removing the label-aware attention raises auxiliary J2J to 0.107; removing the hierarchical encoder hurts all three tasks (body J2J 0.023 → 0.026, auxiliary 0.068 → 0.096, skinning Def-L1 0.60 → 1.88).

• Each geometric component is individually load-bearing. Removing TAMB fusion moves body J2J/B2B from 0.023/0.011 to 0.030/0.015; removing the bone projection loss gives 0.031/0.017; removing chain-to-bone decomposition raises skinning L1 from 7.70 to 11.24 and Def-L1 from 0.60 to 0.84.

• Blind artist evaluation on 50 production assets. Five professional technical artists rated rigs on a 5-point Likert scale with high agreement (ICC(2,k) = 0.95). Among the results visible in the provided content, Pinocchio scored 0.142 body J2J, 0.274 auxiliary J2J, 0.010 skinning L1, quality 2.23, and an estimated 3.48 hours of repair; Puppeteer scored 0.082 on body J2J. The remaining rows, including TailorRig's own in-the-wild scores, are cut off in the supplied text and are not reported here.

Methodology in Plain English

Shape and label encoders. The system first turns the input mesh into a continuous 3D feature field using a PartField-style encoder. It queries that field at 512 farthest-point-sampled locations to get a compact global summary of the character, and at every mesh vertex to get dense local geometry. Each auxiliary category label is mapped to a learnable embedding vector.

A shared hierarchical skeleton encoder. Skeletons are encoded at two levels at once: individual joints (embedded with Fourier features and refined with mesh context) and kinematic chains — the maximal unbranched paths between branching joints, such as "spine," "left arm," or "skirt-left." Fusing the two levels means each joint representation carries both local geometry and chain-level structure. The same architecture, with different parameters, is reused for body alignment, auxiliary generation, and skinning.

Stage 1a — aligning the artist's body template. An arbitrary user template is first canonicalized into a standard T-pose and normalized geometrically. A Topology-Aware Mesh-Bone (TAMB) decoder then applies repeated layers of cross-attention from bones to mesh, followed by EdgeConv over the bone connectivity graph, so the template's topology is preserved while its joints are fitted to the mesh. Training combines an L1 position loss, a semantic classification loss over 11 body classes, and a bone projection loss that projects each predicted joint onto the ground-truth bone segment and penalizes lateral deviation — letting joints slide along the bone axis while blocking sideways drift.

Stage 1b — generating auxiliary bones from labels. Rather than using a label as a static global tag, the system first cross-attends mesh tokens and body-skeleton features against each other, then uses the label embedding as a query to probe both aligned feature sets. This yields two geometry-aware label tokens describing where the category lives on the mesh and on the skeleton. These form a conditioning prefix for an autoregressive decoder (OPT-350M backbone) that reads skeleton trees serialized by depth-first traversal with quantized joint coordinates and emits the auxiliary bone sequence, trained with next-token prediction plus the 21-class semantic loss.

Stage 2 — chain-to-bone skinning. The full skeleton is partitioned into K kinematic chains. Vertex and chain features interact through 4 rounds of reciprocal cross-attention (the coarse stage), then vertex and bone features interact through 8 rounds (the fine stage) — fewer rounds at the coarse level because chain associations are simpler. Chain weights are predicted from a learned cross-attention similarity combined with a geodesic-distance prior. The bone decoder is conditioned on the predicted parent-chain weight, concatenated into each bone's feature, so bone-level competition is effectively confined within each vertex's assigned chains. The skinning loss combines KL divergence on per-bone weights, a vertex-level deformation error under 3 random LBS poses, coarse chain-weight supervision, and the 21-class semantic loss.

Training and data curation. Meshes are resampled to 16,384 points and normalized to a unit ball; a 90%/5%/5% split is used with random augmentation; feature dimension D = 512; AdamW with learning rate 1e-5 and weight decay 0.05; skeleton and skinning modules trained independently. Human-Mix is assembled from Articulation-XL-v2 (48,635 raw → 22,533 curated, 4,060 with auxiliary bones), HumanRig (11,434, adopted in full, 0 auxiliary), an in-house VRoid collection (3,404, of which 3,337 have auxiliary bones), and ModelResource (2,703 → 717, 427 auxiliary). Cross-source duplicates are shortlisted by vertex and bone counts and confirmed manually when mean post-ICP vertex error is below 0.001; the same procedure is applied against a Mixamo reference library, and matched assets are excluded from training. Mixamo assets appear only in qualitative figures.

Why This Matters

Impact on research. The paper argues that the field's framing of rigging as geometry-to-rig recovery is the shared root of two recurring failures — implicit, uncontrollable auxiliary bones and skinning ambiguity under dense skeletons. Reframing the task around explicit control inputs (template + labels) and structural priors (semantics + kinematic chains) gives the community a different problem statement, not just a better number. The chain-to-bone decomposition is a general idea that could be transferred to any dense-skeleton skinning setting.

Real-world applications:

• Game studios populating large worlds with many character variants, where animation libraries must be reused across assets via a chosen body template.
• Film and virtual production pipelines with hero garments — layered skirts, capes, ribbons — that need bespoke auxiliary structures authored to a studio standard.
• Generative 3D asset pipelines, which the paper cites as the motivation for scalable automated rigging.
• Avatar and virtual-try-on systems, where garment-specific bones determine whether cloth follows the body plausibly rather than sticking to limbs.

Industry relevance. The evaluation is explicitly production-oriented: blind ratings from five professional technical artists, a 5-point "broken to production-ready" scale, and an estimate of manual repair hours. The two control axes map directly onto how studios already work — a standard body skeleton for animation reuse plus per-garment decisions about what auxiliary bones to add. The dataset release plan (full rigged FBX for in-house VRoid assets, annotations plus source identifiers for third-party characters) is designed to be license-compliant for commercial research use.

Future Directions

• Closing the controllability evaluation gap. Only one requested category per test character is evaluated; whether artists can request many categories simultaneously — and how off-label errors compound — is not reported in the available content.

• Extending beyond the 21-class taxonomy. The taxonomy (11 body, 10 auxiliary classes) was co-designed with artists, but garment variety is open-ended; how the system handles categories outside the fixed set is an open question.

• Robustness on production-grade rigs. The in-the-wild set spans 52–128 total bones and 2–36 auxiliary bones per character; whether performance holds at much higher bone densities, or on non-humanoid or highly stylized characters, is not established here.

• Dependence on the semantic classifier. The ablations show removing semantic supervision degrades both auxiliary placement and skinning noticeably, so improving or replacing that grounding — for instance with weaker or noisier annotations at scale — is a natural next step.

Target Audience

Technical artists and rigging leads evaluating automated tools for production, and computer vision / graphics researchers working on character rigging, skinning, and structure-aware generative models. It is also relevant to graphics engineers building automated asset pipelines who need to understand the trade-offs between template-based compatibility and template-free flexibility. Readers without background in skeletal animation and transformer architectures will find the method sections demanding.

Authors’ abstract

Humanoid rigging for modern characters with complex garments remains challenging due to the need for dense auxiliary skeletons and precise skinning control. Existing automatic methods primarily recover rigs from geometry, providing limited artist control over auxiliary articulation and often lacking explicit semantic or kinematic-chain structure for dense garment bones, leading to instability and weight bleeding. We present TailorRig, a framework that reformulates garment-aware humanoid rigging as a structured, controllable full-rig generation problem. This formulation shifts the goal from automatic rig recovery to artist-driven rig authoring, where both body-template compatibility and auxiliary articulation are explicit design choices. TailorRig exposes two explicit axes of artist control within a unified framework: a user-specified body skeleton template that preserves body-rig compatibility, and artist-provided semantic labels that drive an autoregressive generator to synthesize requested category-specific auxiliary bone chains. To robustly assign skinning weights under dense skeletal configurations, we further introduce a hierarchical chain-to-bone formulation: vertices are first associated to a kinematic chain, and bone weights are then refined within that chain, reducing weight bleeding between spatially adjacent but semantically unrelated structures. We evaluate TailorRig on a diverse set of characters and introduce Human-Mix, a large-scale dataset of over 38,000 humanoid models curated for garment-aware rigging. Experiments show that TailorRig improves skeleton accuracy, deformation stability, controllability, and practical usability over existing methods.