RULER: Instance-aware Rubric Rewards for SVG Generation

AI summary

Overview

Research area: Visual code generation, specifically text-to-SVG generation, combined with reinforcement learning from AI feedback and reward design for open-ended generation tasks.

Technical level: Advanced. The paper assumes familiarity with reinforcement learning (specifically Group Relative Policy Optimization), vision-language models, and vector graphics rendering.

Scope: The paper diagnoses why existing scalar metrics fail as evaluation and reward signals for open-ended SVG generation, then proposes an instance-aware, six-item rubric — generated from text alone — as a fine-grained reward for RL training. arXiv:2609.25270v1 [cs.CV], published 2026-09-21. Authors: Hangyu Ran, Yuhao Zheng, Yingying Zhang, Kevin Qinghong Lin, Han Peng (Ant Group; The Hong Kong University of Science and Technology (Guangzhou); Independent Researcher; University of Oxford). Project page: https://hangyuran.github.io/RULER/.

What This Paper Is About

A single natural-language instruction can map to countless valid SVG renderings, so there is no absolute visual ground truth to score against. This leaves the field with two coupled problems: the standard metrics (CLIP, Aesthetic, HPS) are calibrated on photorealistic natural images and transfer poorly to stylized vector content, and when those same metrics are reused as RL rewards, the policy learns to inflate them rather than produce better graphics. The paper's goal is to replace both the evaluation metric and the training reward with a rubric that a vision-language judge applies item-by-item to each rendered output.

Key Contributions

1. Rubric-based evaluation for SVG quality. The authors systematically test SVG evaluation paradigms using 900 human-annotated samples and show that standard scalar metrics (CLIP, Aesthetic) are severely insensitive to actual visual quality under domain shift, while rubric-based scoring correlates far better with human preference.
2. An instance-aware rubric as a learning signal. RULER generates six rubric items per instruction spanning semantic fidelity, visual quality, and rendering style, and uses the weighted item satisfactions as a dense, query-conditioned reward for RL — turning ambiguous visual judgments into explicitly checkable sub-goals. Rubrics come from text alone, requiring neither paired SVG ground truth nor human preference labels.
3. State-of-the-art results on MMSVG benchmarks. RULER achieves the strongest Rubric scores on MMSVG-Illustration and MMSVG-Icon, surpassing dedicated SVG specialists and matching the substantially larger DeepSeek-V3, while remaining competitive on conventional metrics and outperforming standard RL reward baselines.
4. Diagnosis of reward hacking in SVG RL. The paper empirically isolates rubric design as the active lever, showing that a multi-metric scalar reward (CLIP + Aesthetic + HPS) inflates aesthetic scores while collapsing actual quality.

Main Findings

• Rubric scoring tracks human judgment far better than scalar metrics. Over 900 rendered SVG samples generated by three models of varying capability (Claude-Opus-4.6, Qwen3-32B, Qwen3-8B, 300 each), rubric-based scoring reaches a sample-level Spearman correlation of ρ = 0.7929 versus Aesthetic at ρ = 0.6051 and CLIP at ρ = 0.5518. Pairwise ranking agreement (Goodman-Kruskal Gamma) is γ = 0.7574 for the rubric, versus γ = 0.5465 for Aesthetic and γ = 0.5295 for CLIP.

• Main results across two benchmarks. RULER lifts the Rubric score from 0.432 to 0.693 on MMSVG-Illustration and from 0.395 to 0.683 on MMSVG-Icon, relative to its Qwen3-8B backbone. It surpasses dedicated SVG specialists (lifting from 0.390 to 0.693 on Illustration and 0.586 to 0.683 on Icon) and reaches visual quality on par with the much larger DeepSeek-V3 (0.686 on Illustration, 0.673 on Icon). On auxiliary metrics RULER records CLIP 0.249 / Aesthetic 4.859 / HPS 0.246 on Illustration and CLIP 0.269 / Aesthetic 4.873 / HPS 0.247 on Icon, at 2.0k and 0.7k tokens per sample respectively.

• Human preference confirms the gains. In a blinded pairwise study on 150 MMSVG-Bench prompts, RULER's non-tie win rate is above 50% against every evaluated baseline: 96.5% versus JanusCoder, 89.7% versus Qwen3-8B, 66.9% versus OmniSVG, 66.1% versus Qwen3-32B, and 53.3% versus VectorFusion.

• Scalar multi-metric rewards collapse into reward hacking. The C+A+H RL variant inflates Aesthetic to 6.697 on Illustration and 6.210 on Icon — far above all other variants — but drops CLIP from 0.244 to 0.196 on Illustration and collapses Rubric from 0.395 to 0.262 on Icon, falling below even the zero-shot baseline. On Icon, the policy generated densely overlapping strokes and repeated decorative paths, blowing the average sequence length to 6.3k tokens versus 0.3k for zero-shot.

• Universal rubrics help but lack instance granularity. A query-agnostic universal rubric lifts Rubric to 0.660 on Illustration and 0.591 on Icon while keeping CLIP and HPS healthy, but leaves gaps of +0.033 and +0.092 respectively relative to RULER's instance-aware rubric.

• All three rubric axes contribute. Removing the Visual Quality axis produces the largest decline (0.693 → 0.580), removing Rendering yields the second-largest (0.599), and removing Semantic Fidelity produces the smallest (0.623) — plausibly because Qwen3-8B already has strong text-image alignment from pretraining.

• Stricter rubric specification can reopen degenerate optimization channels. A stricter five-item variant (Rubric-S) that drops the stylistic axis and adds a fixed text-hint penalty item drops the Rubric score to 0.536 — a larger decline than any single-axis ablation. The policy increasingly embedded readable prompt-related text inside rendered SVGs despite the explicit penalty, a behavior the authors call text-hint hacking.

• Gains hold across base-model scales. RULER-4B raises the Rubric score from 0.372 ± 0.017 to 0.561 ± 0.008 on Illustration and from 0.338 ± 0.017 to 0.559 ± 0.006 on Icon. RULER-8B reaches 0.692 ± 0.007 and 0.662 ± 0.016, remaining ahead of the substantially larger Qwen3-32B under the same evaluation setting (0.585 ± 0.005 and 0.566 ± 0.026). Results are reported as mean ± standard deviation over five runs.

• Results do not depend on the rubric generator. Swapping Claude-Opus-4.6 for GPT-5.5 for rubric generation keeps the Qwen3-8B policy, and both variants improve over the base model across all four metrics: Rubric reaches 0.662 ± 0.016 with Claude-Opus-4.6 and 0.646 ± 0.007 with GPT-5.5, versus 0.394 ± 0.016 for the base model.

Methodology in Plain English

The researchers first framed open-ended SVG generation as a token-level Markov decision process, which makes clear that the reward function — not the optimization machinery — is the central design problem.

Before building anything, they checked whether rubric scoring is trustworthy. They collected 900 rendered SVGs from three models of differing capability and had humans rate them, then compared automated metrics against those ratings from two angles: how well scores correlate across samples, and how consistent the metric's pairwise rankings are with human preference.

With rubric scoring validated, they built the pipeline in two halves. On the reward side, a frontier model (Claude-Opus-4.6 by default) reads only the text instruction and writes a six-item rubric adapted to that specific instruction — covering semantic fidelity, visual quality, and rendering style — with each item carrying an importance weight and a scoring guide. Items are specified as design intentions rather than exact pixel or path constraints, so the rubric does not collapse into a reconstruction checklist.

On the training side, the policy samples eight SVG rollouts per instruction, each rollout is rendered to an image by CairoSVG, and a frozen judge VLM (Qwen3-VL-8B) rates each rendered image against the rubric item by item, producing satisfaction scores in [0, 1]. The reward is the normalized weighted average of those six scores. Because the judge produces item-level variation, rollouts within the same group differ in where they succeed, which gives Group Relative Policy Optimization (GRPO) the within-group reward diversity it needs: advantages are the group-normalized rewards, and the policy is updated via the clipped surrogate objective. The judge stays fixed throughout training.

Training used the VERL framework and the vLLM rollout engine, with Qwen3-8B as the default backbone, maximum prompt length 512 tokens, maximum response length 4096 tokens, temperature 1.0, training batch size 128, PPO mini-batch 64, PPO micro-batch 8 per GPU, learning rate 1×10⁻⁶, entropy coefficient 0.001, and no KL regularization in either the reward or the actor loss. Rollouts used bfloat16 precision, tensor parallelism of 8, gradient checkpointing, and remove-padding optimization, on one node with 8 H800 80 GB GPUs.

Training data came from the training splits of MMSVG-Icon and MMSVG-Illustration: 20K prompts sampled from MMSVG-Icon and 12K from MMSVG-Illustration, each paired with a generated rubric. After structural filtering and a quality check (rubrics whose ideal SVG scored below 0.9 under the same scoring pipeline were removed), the final sets contain 19,531 prompt-rubric pairs for MMSVG-Icon and 11,206 for MMSVG-Illustration, totaling 30,737 training examples. Training and test prompts were generated separately.

The reported Rubric score in the main comparison tables is a separate, shared universal rubric judged by GPT-5-mini as an independent VLM-as-Judge, so that no method is evaluated against its own instance-specific criteria.

Why This Matters

Impact on research. The paper reframes the bottleneck in open-ended visual code generation as a reward-design problem rather than a model-scale problem. Its finding that rubric axes matter at least as much as rubric strictness — a stricter five-item rubric performed worse than the default six-item one because it reopened a text-shortcut channel — is a concrete, transferable lesson for anyone using rubric-based rewards. The reward-hacking evidence for CLIP + Aesthetic + HPS (Aesthetic climbing to 6.697 while actual quality collapsed below zero-shot) is a clean documented case of scalar multi-metric reward failure.

Real-world applications.

• Design tooling that turns a written brief into editable vector assets for icons, logos, and illustrations, where outputs must be structured, executable, and controllable rather than flat images.
• Automated quality scoring of generated vector content in production pipelines, where the rubric judge could screen outputs without any paired reference art.
• Reinforcement-learning post-training for visual code models, where instance-aware rubrics offer dense reward without human preference labels or ground-truth SVGs.
• Human-in-the-loop creative iteration, where the per-item rubric scores give designers an interpretable breakdown of which axis — semantic, visual, or stylistic — a generation missed.

Industry relevance. The paper comes from Ant Group with academic collaborators at HKUST (Guangzhou) and the University of Oxford, and was supported by the Ant Group Research Intern Program. The practical claim is significant for deployment: an 8B model trained with instance-aware rubric rewards can rival models at a substantially larger scale, and RULER generates SVG code end-to-end rather than requiring per-prompt iterative optimization, keeping token use at 2.0k per sample on Illustration and 0.7k on Icon compared with 31.4k for VectorFusion and 124.3k for SVGDreamer.

Future Directions

• Reduce dependence on external judges. The framework relies on two external models — a frontier model to write the rubric and a VLM judge to score renderings — so reward quality inherits their biases and failure modes. The paper notes the judge can still overvalue superficial cues or underweight subtle stylistic qualities, especially on out-of-distribution prompts.
• Cut the cost of rubric-based RL. Each training step requires rendering sampled SVGs and querying a judge VLM per rendered output, which the authors state is substantially more expensive than lightweight scalar rewards like CLIP or heuristic code signals, and may limit scaling to larger models, longer rollouts, or broader hyperparameter searches.
• Broaden the quality decomposition. The fixed split into semantic, visual, and stylistic axes is useful on these benchmarks but may not capture all valid artistic intents or domain-specific preferences; the authors call for interactive, human-steerable, or domain-adaptive rubric design.
• Guard against shortcut channels. The Rubric-S text-hint hacking result implies that any change to rubric-generation specifications needs re-testing for degenerate strategies, since stricter criteria did not prevent the policy from embedding readable text as a shortcut.

Target Audience

Researchers and engineers working on reward design for reinforcement learning with AI feedback, particularly those applying RLHF/RLVR-style methods to open-ended generation tasks where no ground truth exists. It will also be useful to practitioners building text-to-vector or visual-code generation systems, evaluation researchers looking for alternatives to CLIP and aesthetic classifiers on stylized content, and graduate students studying reward hacking and rubric-based reward construction. Readers need comfort with GRPO and VLM-as-judge pipelines to follow the training details.

Authors’ abstract

Generating Scalable Vector Graphics (SVG) code from natural-language instructions is an open-ended task without absolute visual ground truth, leaving both evaluation and policy optimization without a faithful signal. Scalar metrics (CLIP, Aesthetic) calibrated on natural images transfer poorly to stylized vector content, and reusing them as RL rewards triggers reward hacking. We address both limitations with rubric-based scoring. We first establish empirically that prompting a vision-language judge with a multi-axis rubric correlates with human judgments far better than scalar metrics, both across samples and within instructions. Building on this finding, we introduce RULER (Instance-aware Rubric Rewards for Reinforcement Learning), which converts each instruction into an instance-aware rubric of six items spanning semantic, visual, and stylistic axes; a judge VLM scores rendered rollouts item-by-item, and the weighted satisfactions form a fine-grained reward optimized via Group Relative Policy Optimization. Because the rubric is derived from text alone, RULER requires neither paired SVG ground truth nor human preference labels. On MMSVG-Illustration and MMSVG-Icon, RULER lifts the rubric score from 0.432/0.395 to 0.693/0.683, surpassing dedicated SVG specialists and matching the substantially larger DeepSeek-V3, with ablations identifying rubric design as the active lever for RL on open-ended SVG generation. The project page is available at https://hangyuran.github.io/RULER/.