Paint-Anything: Unified Any-Color Control for Image Generation and Editing

AI summary

Overview

Research area: Computer Vision, specifically controllable image generation and image editing with diffusion models.

Technical level: Intermediate. The core idea (using hex color codes as prompts) is easy to grasp, but the training recipe involves details about noise timesteps and diffusion-based supervision that assume some familiarity with generative image models.

Scope: This paper introduces a single hex-code-driven interface that lets users specify the exact target color of a named object for both image generation and image editing, and it provides a large training dataset plus a new benchmark for measuring how accurately those colors are reproduced.

What This Paper Is About

Professional design workflows require precise color control: a designer needs to say "make this chair #3B7A57" and get exactly that color, whether they are generating a new image or editing an existing one. Prior systems handle generation, editing, and colorization separately, and often require special color encodings or custom inference tricks rather than a simple, shared way to specify a color. This paper's goal is to unify those tasks behind one plain hex-value prompt interface, so the same input format works across both generating and editing, at the level of individual objects.

Key Contributions

1. A unified hex-prompt interface. Paint-Anything learns a single shared interface where a 24-bit hex value controls an object's color for both image generation and image editing, rather than using task-specific color representations or separate models.

2. The Paint-500K data pipeline and dataset. The authors build a pipeline that derives training data from real images through object grounding (locating objects), perceptual color labeling (assigning color names/values), and synthesis of editing pairs, producing a 500K-scale dataset.

3. Pure-color anchors for exact supervision. Because real photos contain shadows and lighting, their color labels are only approximate. The paper adds synthetic pure-color anchors whose pixels match their paired hex values exactly, and applies these only at high-noise timesteps while reserving low-noise training for natural images.

4. Any Color Benchmark (ACBench). A new benchmark with two parts, ACBench-T2I (text-to-image) and ACBench-Edit (editing), designed to measure object-level hex color fidelity in generated and edited outputs.

Main Findings

• Large gains in color fidelity: On FLUX.2-4B, Paint-Anything improves ACBench-T2I scores by 85.3% and ACBench-Edit scores by 28.3% over the base model.
• Best comparative color performance: It achieves the highest average CompColor score among all compared methods, indicating strong color accuracy relative to prior approaches.
• The training recipe matters: Ablation studies support the design choices in the training setup, including the combination of real-image supervision with pure-color anchors.
• Timestep separation is effective: Restricting exact-color anchors to high-noise timesteps and leaving low-noise training to natural images appears to be a workable way to reconcile exact color targets with realistic shading and texture.

Methodology in Plain English

The researchers start from a simple observation: modern language models already understand what a hex code like #FF5733 means, so a color can be communicated as plain text rather than through a special-purpose color encoder. They build a training dataset by taking real photographs, identifying objects in them, labeling the objects' colors, and synthesizing before/after pairs that simulate color edits. Real photos, however, have shadows and highlights, so the labeled color is never exactly the object's pixel value. To fix that, they generate synthetic objects filled with a single flat color that exactly matches a given hex code, and use these only during the noisiest stage of training, when the model is learning broad structure and color identity. During the later, low-noise stages, the model trains on real images so it learns realistic lighting and texture. A single model is then trained to accept a hex value plus an object reference and apply that color, whether the image is being created from scratch or modified.

Why This Matters

Impact on research. The work argues that color control does not require bespoke color encodings or task-specific inference pipelines; a shared text-based hex prompt suffices when paired with the right supervision. It also contributes a dataset and a benchmark, which gives the community a standardized way to measure object-level color accuracy, a property that prior image-generation evaluations largely ignored.

Real-world applications.

• Graphic and brand design: Applying exact brand palette values to products, logos, or mockups without manual recoloring.
• E-commerce and product visualization: Generating or editing product images in every available colorway from a single source image and a list of hex codes.
• Interior design and architecture: Letting clients preview furniture, walls, or finishes in precise manufacturer colors.
• Game and 3D asset prototyping: Quickly exploring color variants of characters and props under a locked palette.

Industry relevance. Any industry that depends on color accuracy—apparel, automotive, cosmetics, packaging, advertising—has a direct stake in tools that turn an arbitrary color specification into a faithful visual result. A method that improves fidelity by large margins over a strong base model suggests this capability is becoming practical for production pipelines rather than remaining a research demo.

Future Directions

• Extending beyond flat object color: Handling gradients, patterns, textures, and metallic or translucent materials, where "the color of this object" is not a single value.
• Multi-object and relational control: Assigning different hex values to several objects in one image while preserving their interactions and scene lighting.
• Generalizing to video and 3D: Applying the same hex-prompt interface to video editing or 3D asset generation, where color consistency must hold across frames or viewpoints.
• Reducing dataset dependence and bias: Testing whether the pipeline transfers to domains poorly represented in real-image data, and whether the pure-color anchor trick scales to more complex, non-diffusion generation architectures.

Target Audience

This paper is most useful to researchers and engineers working on controllable image generation and editing, particularly those interested in attribute-level conditioning, diffusion training recipes, and evaluation benchmarks. Practitioners building design, e-commerce, or product-visualization tools will find the capabilities and benchmark results directly relevant, while graduate students entering the field can use it as a clear example of how data construction, supervision design, and benchmarking fit together in a modern generative-model paper.

Authors’ abstract

Professional design requires any-color control: the ability to specify an object's target color with any 24-bit hex value for image generation and editing. Prior work has explored color generation, editing, and colorization, but often relies on dedicated color representations or specialized inference procedures. Advances in large language models offer a simpler starting point: even compact models can associate hex values with color semantics. We present Paint-Anything, which learns a shared hex-prompt interface for generation and editing through object-level color supervision. We develop a data pipeline that constructs Paint-500K from real images through object grounding, perceptual color labeling, and editing-pair synthesis. Since shadows make real-image labels only approximate colors, we complement this supervision with pure-color anchors whose pixels exactly match their paired hex values. These anchors are used only at high-noise timesteps, leaving low-noise training to natural images. We further introduce Any Color Benchmark (ACBench), comprising ACBench-T2I and ACBench-Edit, to measure object-level hex color fidelity across both tasks. On FLUX.2-4B, Paint-Anything improves ACBench-T2I and ACBench-Edit scores by 85.3% and 28.3%, respectively, relative to the base model, with ablations supporting the training recipe. It also achieves the highest average CompColor score among the compared methods.