AesCode: Aesthetic Code Generation with Decoupled Cross-Modal Rewards

AI summary

Overview

Research area: Multimodal generative AI, specifically reinforcement-learning post-training of vision-language models that emit code (HTML/CSS) for information-rich visual artifacts such as slides, posters, dashboards, and webpages.

Technical level: Advanced. The paper assumes familiarity with vision-language models, reinforcement learning from verifiable rewards, GRPO-style group-normalized policy optimization, and multi-objective reward shaping.

Scope: The paper introduces a training and evaluation framework in which a code-generating model is conditioned on both a text prompt and an image-generator reference, supervised by design-graph-derived rewards split across deterministic verifiers and a graph-bound VLM rubric judge.

What This Paper Is About

Code models can produce structured, editable, verifiable artifacts, but they cannot "see" the canvas, so their layouts and color choices are often visually dated. Image generators produce striking designs but frequently misrender text, numbers, and logical relations, and their raster output cannot be edited or verified. The paper's goal is a setting where an image generated from the same prompt acts only as an aesthetic prior, while the code model still has to satisfy the prompt's content exactly.

Key Contributions

1. A new paradigm for aesthetic code generation. An image generated from the same prompt is used as an aesthetic prior: it conditions the code model as input and supplies visual training signals, rather than serving as a ground-truth target for pixel-level reconstruction.

2. AesCode with decoupled cross-modal rewards. The target is represented as a design graph, from which the method derives deterministic verifiers for properties parsable from code and rendering, plus a sample-specific Visual Graph Rubric (VGR) answered by a VLM for non-parsable properties. The seven channels are balanced with Group reward-Decoupled Normalization Policy Optimization (GDPO).

3. An infographic benchmark with separated scoring. A 300-prompt benchmark scores verifiable correctness (Rule) and visual quality (Visual) separately, with automated visual scores reported as closely matching human judgments.

4. Empirical evidence that references help broadly. Ablations show reference images improve visual quality for every tested model, both as model input and as a source of visual rewards.

Main Findings

• Large gains over same-scale backbones. Relative to same-scale Qwen3-VL baselines given the same references, AesCode-8B and AesCode-32B improve visual quality by 31.1 and 22.4 percentage points. Overall rises from 51.93 to 82.94 at 8B and from 61.10 to 85.89 at 32B.

• Comparable visual quality to frontier closed-source models. With references, AesCode-8B reaches a Visual score of 75.80 and AesCode-32B reaches 77.44, versus 76.76 for GPT-5.5 and 77.23 for Claude Opus 4.8. Overall: AesCode-32B 85.89, AesCode-8B 82.94, GPT-5.5 81.28, Claude Opus 4.8 80.39.

• Stronger verifiable correctness. AesCode-32B leads the Table/Chart check at 90.37 and reaches Rule 94.33 versus 85.80 for GPT-5.5 and 83.55 for Claude Opus 4.8 under reference conditioning. The paper states AesCode matches frontier systems visually while making noticeably fewer content errors.

• References improve every model tested. Closed and open models all gain visual presentation when given reference images, but Style remains a shared bottleneck below 60% for all systems.

• Transfer to external benchmarks. AesCode-32B improves on its Qwen3-VL-32B backbone on all fourteen measures across GDB/lica, SlidesBench, and Design2Code, with the largest gains on design-sensitive measures (SlidesBench layout and color; GDB/lica NIMA, FID, and color harmony). Gains are smaller on Design2Code, which the paper describes as near saturation and less aligned with human aesthetic judgments; GPT-5.5 remains stronger in exact screenshot fidelity.

• References internalized during training. Removing the reference only at inference reduces AesCode-8B Visual by 1.00 point, versus 19.55 for its backbone and 10.04 for GPT-5.5. Even when the policy never sees a reference in training or inference, reference-based rewards raise prompt-only Visual from 25.17 to 69.71 — 5.09 points below reference-conditioned training under the same inference setting.

• RL drives most of the gain, SFT improves initialization. Under the same RL budget, Overall rises from 77.97 to 82.94 when SFT is included.

• GDPO beats summed rewards. GDPO adds 5.12 Visual points over scalar GRPO by normalizing channels independently.

• Design supervision determines visual quality. Code correctness alone yields 66.59 Visual; adding Layout/Whitespace raises it to 68.12. CLIP reaches 70.49, a holistic VLM score reaches 68.91, and VGR reaches 75.80 without sacrificing Rule. Authoring the visual questions directly without the design graph lowers Visual to 72.61 at essentially unchanged Rule (90.07 versus 90.05).

• Human validation. Human-majority judgments agree with the training VGR and the Visual checklist on 89.3% and 93.3% of sampled predicates. AesCode-32B achieves higher Content satisfaction than GPT-5.5 (86.50% versus 83.47%) with equivalent aesthetic quality (mean difference −0.15, 95% CI [−0.34, 0.03], within a pre-specified ±0.5 margin). The Visual score agrees with direct human pairwise choices in 80.66% of comparisons.

• Recurring failure modes. Canvas overflow (rare after training), lost relational structure, and semantic elements degrading into generic marks.

Methodology in Plain English

The pipeline starts from roughly 18,000 human-designed slides, posters, dashboards, and webpages, clustered by CLIP image features and sampled in a stratified way so peripheral designs are covered alongside typical ones. GPT-5.4-mini infers a content-only prompt for each sampled image — specifying artifact type, required text, chart and table data, major layout zones, and reading flow, but no specific colors, shapes, spacing, or typography.

For cold-start supervised fine-tuning, GPT-5.5 is given the prompt plus the original human design and writes three HTML candidates; these are rendered and scored by the reward system, and the top candidate is kept if its reward exceeds 0.85. This yields 3,000 demonstrations. For reinforcement learning, the original images are replaced with synthetic references from GPT-Image-2, the prompt pool is rebalanced across clusters to keep usable learning signal, producing roughly 8,000 prompts.

The model is trained to produce code that is compared against a target design graph. That graph has a canvas root plus nodes for atomic elements (text, images, charts, tables, badges) and containers (regions, cards, panels, lists, groups), with each node carrying decoupled content and visual attributes and edges carrying relations such as containment, alignment, order, adjacency, and semantic connection.

Rewards come in two families. Six deterministic verifiers score what a program can decide: execution and fatal errors, required text, canvas boundary and aspect ratio, table/chart structure and data, semantic layout agreement between OCR blocks in reference and candidate, and whitespace distribution. The seventh channel is a VLM judge answering a sample-specific set of yes/no questions derived from the design graph, with the pass rate as the score.

Each RL step samples 128 prompts with 8 rollouts each; the 8B and 32B models train for 400 and 520 steps. SFT uses a learning rate of 10⁻⁵ and GDPO uses AdamW at 5×10⁻⁶. Because the seven channels differ in scale, sparsity, and saturation, GDPO normalizes each channel separately before averaging, with masks marking which channels apply to a given rollout. Evaluation is deliberately separated from training: Rule averages Text, Boundary, and Table/Chart, while Visual averages Content, Layout, and Style checklists averaging 35.8 questions each, with each system receiving identical sample-specific criteria.

Why This Matters

Impact on research. The paper reframes image generators as scalable teachers of visual design rather than as content generators, and shows that a noisy, sometimes factually wrong generated image can serve as a useful training signal if the supervision is decomposed into attributable properties. It also offers a concrete demonstration that multi-objective reward normalization matters when verifiable and perceptual objectives learn at different rates.

Real-world applications.

• Automated generation of business slides, decks, and internal reports where content must be exactly right but visual polish matters.
• Marketing and event posters, where a brief specifies text and brand content while an image generator supplies layout and style inspiration.
• Data dashboards that must render real charts and tables (the method constrains charts to ECharts and tables to HTML table structures, so outputs are executable and directly verifiable).
• Design-to-code tooling and website or artifact prototyping where editable markup is more useful than a raster image.

Industry relevance. Editable, verifiable output is directly deployable in content pipelines where raster images are not: generated code can be audited, versioned, and modified. The result that AesCode-8B reaches overall quality comparable to much larger frontier systems also suggests that targeted reward design can substitute for raw scale in this task.

Future Directions

• Pixel-level geometry. Canvas overflow persists because code generation cannot reliably anticipate final pixel geometry; the paper calls for visually grounded iterative refinement on this front.
• Relational structure. Relations are harder than individual elements, and relation-level rewards remain sparse and do not localize which edge failed.
• Semantic visual elements. Icons and similar elements degrade into generic marks because they require fine-grained SVG drawing grounded in meaning.
• More precise structural feedback and better benchmarks. The paper notes that Design2Code-style reconstruction metrics are near saturation and align less closely with human aesthetic judgment, motivating evaluation that better reflects design-sensitive quality.

Target Audience

Researchers and engineers working on multimodal generation, reinforcement learning from verifiable or rubric-based rewards, and code-generating models for visual artifacts. It is also relevant to practitioners building design automation tools, slide and dashboard generators, or text-to-code systems that need outputs which are both editable and visually credible. Readers without a background in RL post-training or vision-language models will find the high-level framing accessible but the reward and optimization details demanding.

Authors’ abstract

Generating information-rich visual artifacts such as slides, posters, and dashboards as code is appealing because the outputs are structured, editable, and verifiable. Yet code models cannot see how their choices of layout and color come together on the canvas, often limiting spatial planning and aesthetic quality. Image generators excel at visual design but can misrender requested content, making their outputs unreliable targets for direct reproduction. We therefore introduce AesCode, which generates code grounded in the content prompt while using an image generated from the same prompt as an aesthetic guidance. To do so, AesCode represents the target as a design graph that describes the whole canvas and makes each property attributable. From this graph, we derive complementary cross-modal rewards: deterministic verifiers score properties parsable from code and rendering, while a VLM judge assesses the non-parsable ones with a sample-specific Visual Graph Rubric (VGR) tied to graph elements and relations. We optimize these decoupled rewards through reinforcement learning with per-channel normalization to jointly improve spatial planning, aesthetic quality, and verifiable correctness. To evaluate this setting, we build a benchmark that scores verifiable correctness and visual quality separately, with visual scores highly consistent with human judgments. We find that reference images substantially improve the visual quality of code generated by existing models, and ablations show that AesCode benefits from references both as model input and as a source of visual rewards. Relative to same-scale Qwen3-VL baselines given the same references, AesCode-8B and AesCode-32B improve visual quality by 31.1 and 22.4 percentage points, reaching scores comparable to those of closed-source frontier models such as GPT-5.5 and Claude Opus 4.8.